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Perception/Action:  An  Holistic  Approach  II 

John  M.  Flach 
Wright  State  University 

Abstract 

This  final  report  reviews  three  years  of  research  focused  on  the 
coordination  of  perception  and  action.  Human  performance  has  been  evaluated 
within  the  framework  of  a  closed-loop  system  where  perception  and  action  are 
intimately  coupled.  Four  problems  have  been  studied:  the  control  of  locomotion, 
dynamic  occlusion,  depth  perception,  and  minimally  invasive  surgery.  Studies 
of  the  control  of  locomotion  have  shown  that  for  control  of  altitude  there  was  an 
interaction  between  the  flow  structure  (splay  or  depression  angle)  and  the  event 
dynamic  (hover  or  forward  flight).  Results  showed  that  in  hover  conditions, 
depression  angle  specifies  altitude  changes  most  reliably;  but  in  forward  flight 
conditions,  splay  angle  specifies  altitude  changes  most  reliably.  These  results  are 
explained  as  a  function  of  singal-to-noise  ratios  within  the  optical  flow  field. 
Initial  work  to  evaluate  control  of  collision  is  also  discussed.  The  dynamic 
occlusion  task  was  used  to  evaluate  effects  of  mode  of  observation  (active  versus 
passive)  on  performance.  This  task  was  chosen  because  of  the  ability  to  control 
for  information  differences.  Results  showed  that  the  observation  mode  had  little 
effect.  Previous  research  that  has  shown  an  advantage  for  active  observers 
appears  to  be  due  to  information  differences,  not  to  the  observation  mode. 
Research  on  depth  perception  showed  that  difference  that  had  previously  been 
found  for  response  mode  (walking  versus  matching)  could  be  attributed  to  the 
response  frame  (egocentric  versus  exocentric).  Subjects  were  generally  accurate 
(little  effects  due  to  foreshortening)  when  using  an  egocentric  response  frame. 
Finally,  preliminary  work  to  evaluate  performance  in  minimally  invasive  surgery 
is  outlined.  This  work  included  field  observations  of  surgery  as  well  as 
laboratory  studies  to  examine  the  coordination  problems  involved  with  remote 
vision  and  two-dimensional  video  displays. 
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8  Figure  1.  In  studying  coordination  within  a  closed-loop  system  the 

experimenter  manipulates  goal,  action,  and  feedback  constraints, 
rather  than  the  stimulus.  The  stimulus  at  any  moment  is  a  function 
of  the  responses  of  the  subject. 

9  Figure  2.  In  the  pursuit  model  of  control,  behavior  is  jointly  determined  by 

open-  and  closed-loop  paths.  The  open-loop  path  represents 
knowledge  and  expectations.  The  closed-loop  path  represents 
perceptual  monitoring  of  performance. 

10  Figure  3.  A  state  space  representation  illustrating  constraints  that  might 

bound  the  safe  field  of  travel  for  low  altitude  flight.  The  ceiling 
and  ground  represent  goal  constraints  in  terms  of  specifying 
negative  consequences  of  collision  with  the  ground  or  detection  by 
the  enemy.  The  solid  parabolic  curves  represent  the  2nd  order 
dynamic  constraints  on  aircraft  motion.  The  dotted  parabolic 
curves  represent  feedback  constraints  in  terms  of  the  pilot's  ability 
to  resolve  changes  in  altitude  or  velocity. 

15  Figure  4.  An  illustration  of  the  relation  between  splay  angle  and  altitude. 

As  altitude  decreases  from  Z1  to  Z2  the  splay  angle  increases. 

16  Figure  5.  The  change  of  splay  or  depression  angle  for  a  5  ft  decrease  in 

altitude  from  an  initial  altitude  of  25  ft  as  a  fucntion  of  the  initial 

angular  position  is  illustrated  as  a  sine  function  with  peaks  at  +45°. 
The  change  of  splay  and  depression  angle  as  a  function  of  a  5  ft 
fore-aft  or  lateral  displacement  is  illustrated  as  a  cos2  function  with 

peak  at  0°.  The  full  range  of  initial  splay  anles  is  typically  visible  in 
the  frontal  field  of  view.  However,  alll  initial  depression  angles  are 
not  visibel.  For  the  field  of  view  simulated  in  our  experiments 

initial  depression  angles  below  51.87°  were  outside  of  the  field  of 
view. 

18  Figure  6.  Texture  lines  perpendicular  to  the  forward  path  of  travel  provide 
information  for  change  in  altitude  in  terms  of  depression  angle.  As 
the  observer  moves  from  a  high  altitude  (Zj)  to  a  lower  altitude  (Z2) 
the  depression  angle  becomes  larger  and  the  texture  lines  move  up 
in  the  field  of  view  toward  the  horizon. 

20  Figure  7.  Three  types  of  ground  textures  that  have  been  used  to  isolate  the 
components  of  optic  flow  associated  with  change  of  altitude. 

24  Figure  8.  RMS  altitude  error  as  a  function  of  texture  and  forward  flow  rate. 

Note  that  for  the  hover  condidtion  less  error  is  seen  with 
depression  texture,  but  with  forward  motion  splay  texture  results 
in  less  error. 

25  Figure  9.  Correlated  control  power  as  a  function  of  texture  type,  disturbance 

and  flow  rate,  note  that  the  correlations  with  the  altitude 
disturbance  are  equivalent  for  the  different  textures  in  the  hover 
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condition.  However,  there  is  a  higher  correlation  for  splay  texture 
when  there  is  signigicant  forward  motion. 

26  Figure  10.  The  change  in  depression  angle  as  a  function  of  optical  position 

for  two  movements.  The  dotted  line  shos  optical  change  due  to  a  5 
ft  change  in  altitude  from  an  altitude  of  25  ft.  The  solid  line  shows 
optical  change  resulting  from  a  25  ft  change  (i.ev  one  eyeheight  in 
fore-aft  position. 

27  Figure  11.  These  diagrams  illustrate  the  consequences  of  the  two  different 

dynamic  constraints.  For  the  constant  ascent  rate  dynamic,  ascent 
rate  is  constant.  Thus,  to  just  clear  the  cliff  the  pilot  must  initiate 
the  climb  farther  from  the  cliff  when  the  forward  velocity  is  greater 
(a)  than  when  it  is  lower  (b).  For  the  proportional  ascent  rate 
dynamic,  ascent  rate  is  proportional  to  forward  velocity.  The  result 
of  this  is  that  the  pilot  must  initiate  ascent  at  the  same  position, 
independent  of  whether  forward  velocity  is  high  (c)  or  low  (d). 

27  Figure  12.  The  solid  line  represents  optimal  performance  for  the  constant 
ascent  rate  dynamic.  The  dotted  line  represents  optimal 
performance  for  the  proportional  ascent  rate  dynamic.  The  fill  pH 
circles  show  actual  obtained  performance  for  subjects  trained  with 
the  constant  ascent  rate  dynamic.  The  open  circles  show  actual 
performance  for  subjects  trained  in  the  proportional  ascent  rate 
dynamic. 

32  Figure  13.  The  configural  atitude  display  integrates  pitch,  roll,  altitude,  air 
speed  and  heading  into  a  single  configural  representation. 

41  Figure  14.  Perspective  and  schematic  views  of  a  low  fidelity,  laboratory 

simulation  for  studying  the  perceptual-motor  demands  of 
laparoscopic  surgery.  The  subject's  task  is  to  pick-up  plastic  chips 
and  to  transfer  them  from  one  container  to  another  using  Endo- 
Babcock  surgical  instruments.  Direct  viewing  of  the  workspace  is 
occluded.  Visual  feedback  is  provided  using  a  video  camera  and 
video  monitor. 

42  Figure  15.  Schematic  view  showing  three  transfer  conditions.  Each  transfer 

condition  maintains  a  different  invariant  relation  to  the  training 
condition  shown  in  Figure  8.  Transfer  A  maintains  an  invariant 
relation  between  the  hands  and  the  work.  Transfer  B  maintains  an 
invariant  relation  between  the  camera  and  the  work.  Transfer  C 
maintains  an  invariant  relation  between  the  camera  and  the  hands. 
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1.0  General  Overview 

* 

Two  strategies  that  scientists  use  to  reduce  the  complexity  of  the  natural 
world  are  reductionism  and  abstraction.  Reductionism  deals  with  complexity  by 
describing  the  system  in  terms  of  a  small  number  of  fundamental  elements.  In 
physics  the  search  for  fundamental  particles  and  in  psychology  the  search  for 
independent  information  processing  stages  or  for  local  neural  mechanisms  reflect 
this  strategy  for  attacking  complexity.  Abstraction  deals  with  complexity  by 
searching  for  global  relational  properties  that  exist  somewhat  independently  of 
the  elements  whose  behavior  they  govern.  In  physics,  the  idea  of  "fields"  (e.g., 
gravitational  or  electro-magnetic)  are  prototypical  examples  of  the  strategy  of 
abstraction.  In  psychology,  the  cybernetic  hypothesis,  that  human  behavior  may 
be  constrained  by  the  same  kind  of  stability  constraints  as  a  servomechanism,  is 
an  example  of  abstraction.  This  is  an  abstraction  in  the  sense  that  within  a 
closed-loop  system  stability  is  not  associated  with  any  particular  element  of  the 
system.  Rather,  stability  depends  on  global  relations  between  forward  loop  gain 
(sensitivity  to  error)  and  time  delays.  Recent  work  using  nonlinear  dynamics  to 
model  phase  transitions  in  rhythmic  behaviors  such  as  walking  also  reflect  the 
strategy  of  abstraction  (e.g.,  Kelso,  1995). 

Both  reductionism  and  abstraction  have  proven  value  as  strategies  of 
science.  However,  some  problems  are  better  approached  using  one  strategy  or 
the  other.  A  fundamental  commitment  of  our  research  program  is  that  situation 
awareness  and  situated  cognition  are  fundamentally  problems  of  coordination  —  in  that 
they  address  the  human  operator's  capacity  to  adapt  to  changing  demands  and  contexts 
in  dynamic  environments.  As  problems  of  coordination,  we  believe  that  they  are 
best  attacked  using  the  strategy  of  abstraction.  While  there  are  a  number  of 
obvious  and  successful  examples  of  how  the  strategy  of  abstraction  can  be  used 
to  address  problems  of  human  performance,  psychology  tends  to  be  dominated 
by  reductionist  strategies.  For  this  reason,  this  final  report  will  begin  with  a  brief 
description  of  our  research  framework.  This  will  be  followed  by  descriptions  of 
our  experimental  programs  in  four  areas:  the  control  of  locomotion,  dynamic 
occlusion,  distance  perception,  and  minimally  invasive  surgery. 

1.1  General  Framework 

Although  psychologists  have  greatly  expanded  how  they  conceive  of 
stimulus  and  response,  the  reductionist  strategy  depends  heavily  on  stimulus- 
response  (SR)  as  the  fundamental  unit  of  analysis.  The  experimenter  controls  the 
stimulus  and  measures  the  subject's  responses.  In  order  to  achieve  control  over 
the  stimulus,  the  experimenter  must  break  all  feedback  links  between  the 
subject's  actions  and  the  stimulus.  The  tachistoscope  was  an  excellent  tool  for 
accomplishing  this  and  the  computer  continues  to  be  used  essentially  as  a 
tachistoscope  in  many  psychological  laboratories  studying  learning,  skill, 
attention,  and  perception.  The  stimuli  to  be  presented  and  the  responses 
measured  are  chosen  to  isolate  a  particular  stage  within  the  human  information 
processing  system. 
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Figure  1.  In  studying  coordination  within  a  closed-loop  system  the  experimenter 
manipulates  goal,  action,  and  feedback  constraints,  rather  than  the  stimulus.  The 
stimulus  at  any  moment  is  a  function  of  the  responses  of  the  subject. 

Since  our  research  focuses  on  the  coordination  between  perception  and 
action  it  is  not  possible  to  rely  exclusively  on  experimental  paradigms  that  cut 
the  feedback  link  that  supports  that  coordination.  We  are  forced,  in  the  study  of 
coordination,  to  relinquish  control  of  the  stimulus  to  our  subjects.  So  how  is  it 
possible  to  do  controlled  experimental  work  if  you  have  no  control  of  the 
stimulus?  Figure  1  illustrates  how  we  approach  this  problem.  We  manipulate 
constraints,  rather  than  stimuli.  As  illustrated  in  Figure  1,  we  distinguish  3 
general  classes  of  constraints.  Constraints  on  action  refer  to  dynamic  constraints 
that  limit  the  possible  responses  or  movements  that  are  possible  within  the 
problem  space  or  work  domain.  For  example,  in  studying  the  control  of 
locomotion  these  constraints  might  reflect  the  dynamics  of  the  vehicle  and 
control  interface  that  are  provided  for  the  human  actor.  Constraints  on 
information,  refer  to  the  properties  of  the  feedback  that  are  made  available  to  the 
actor  for  monitoring  progress  through  the  problem  space.  Again,  in  the  context 
of  control  of  locomotion,  these  constraints  might  reflect  the  optical  properties  of 
dynamic  flow  fields  and/ or  the  display  interface  that  is  made  available  to  the 
human  actor.  Finally,  goal  constraints  refer  to  constraints  on  intention.  This 
includes  the  value  system  that  determines  the  costs  and  benefits  associated  with 
the  problem  space.  Again,  in  the  context  of  the  control  of  locomotion,  these 
constraints  might  include  an  explicit  track  from  which  the  subject  is  to  minimize 
deviations  or  instructions  with  regard  to  speed  and  accuracy. 

The  manipulation  of  constraints,  rather  than  stimuli,  has  important 
implications  for  the  model  of  causality  used  to  explain  phenomenon.  The  SR 
approach  is  based  on  what  Einstein  and  Infeld  (1938)  have  identified  as  a 
mechanistic  view  of  causality.  In  this  view,  stimuli  and  mediating  neural 
mechanisms  are  seen  as  the  "causes"  of  responses.  The  idea  of  relating  responses 
to  constraint,  reflects  what  Einstein  and  Infeld  (1938)  refer  to  as  a  field  view. 
Einstein  and  Infeld  (1938)  explain  that  where  a  mechanistic  view  would  describe 
an  interaction  in  terms  of  one  electrical  particle  acting  on  another,  "in  the  new 
field  language  it  is  the  description  of  the  field  between  the  two  charges,  and  not 
the  charges  themselves,  which  is  essential  for  an  understanding  of  their  action" 
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(p.  157).  In  our  framework,  we  consider  the  action,  information,  and  intentional 
constraints  to  be  critical  components  of  the  ''behavioral  field".  These  constraints 
are  not  deterministic  causes  of  behavior.  Rather,  the  constraints  limit  or  bound 
the  possibilities  for  action.  A  fundamental  goal  of  our  research  program  is  to  discover 
effective  ways  to  represent  these  behavioral  fields  so  that  we  can  better  understand  and 
predict  the  range  of  possibilities  for  action  that  bound  the  adaptive  relationship  between 
humans  and  their  environments. 


Figure  2.  In  the  pursuit  model  of  control,  behavior  is  jointly  determined  by 
open-  and  closed-loop  paths.  The  open-loop  path  represents  knowledge  and 
expectations.  The  closed-loop  path  represents  perceptual  monitoring  of 
performance. 

We  have  found  the  language  of  control  theory  to  be  an  important  tool  for 
visualizing  the  behavioral  field.  It  is  important  to  note  that  the  use  of  control 
theory  is  not  equivalent  to  the  cybernetic  hypothesis  in  which  human  behavior  is 
modeled  in  terms  of  a  simple  servomechanism  (or  TOTE  unit  -  Miller,  Galanter, 
&  Pribram,  1960).  Control  theory  is  a  much  broader  language  that  includes 
open-  and  closed-loop  systems  as  well  as  adaptive  systems.  Our  basic  model  for 
the  human  controller  is  what  Krendal  and  McRuer  (1960)  have  called  the  pursuit 
mode  of  control.  As  shown  in  Figure  2,  the  pursuit  mode  involves  both  open- 
and  closed-loop  paths.  The  open-loop  path  reflects  the  controller's  knowledge  of 
the  action  constraints  (e.g.,  the  pilots  internal  model  of  the  aircraft)  and  goals 
(e.g.,  patterns  within  the  track  to  be  followed).  Based  on  this  knowledge  (e.g., 
expectations  about  the  response  of  the  aircraft),  the  pilot  is  able  to  respond 
directly  to  an  input  command  (i.e.,  intention  or  goal  constraint)  or  to  a 
predictable  input  pattern  and  it  is  not  necessary  to  wait  for  an  error  signal.  To 
the  extent  that  the  pilot's  knowledge  is  applied  appropriately,  the  response  will 
meet  expectations,  achieve  the  goal,  and  little  error  will  result.  However,  if  the 
response  is  not  appropriate,  then  it  is  still  possible  for  the  pilot  to  make 
corrections  using  the  error  signal  in  the  closed-loop  path. 
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RATE  OF  CLIMB  |  RATE  OF  DESCENT 


Figure  3.  A  state  space  representation  illustrating  constraints  that  might  bound 
the  safe  field  of  travel  for  low  altitude  flight.  The  ceiling  and  ground  represent 
goal  constraints  in  terms  of  specifying  negative  consequences  of  collision  with 
the  ground  or  detection  by  the  enemy.  The  solid  parabolic  curves  represent  the 
2nd  order  dynamic  constraints  on  aircraft  motion.  The  dotted  parabolic  curves 
represent  feedback  constraints  in  terms  of  the  pilot's  ability  to  resolve  changes  in 
altitude  or  velocity.  ”  ° 

An  important  control  theoretic  representation  for  illustrating  the 
constraints  of  a  dynamic  system  is  the  state  space.  Mach  and  Warren  (1995)  have 
used  the  state  space  to  represent  the  safe  field  of  travel  for  low  altitude  flight. 
Figure  3  is  adapted  from  Mach  and  Warren.  The  boundaries  in  this  diagram 
illustrate  some  of  the  constraints  for  low  altitude  flight.  The  ground  and  ceiling 
represent  goal  constraints.  That  is,  these  boundaries  are  associated  with  negative 
consequences  (i.e.,  collision  with  the  ground  or  detection  by  the  enemy).  The 
solid  parabolic  lines  represent  action  constraints.  The  shape  of  this  boundary 
reflects  a  second-order  approximation  to  the  aircraft  dynamics.  The  solid 
boundary  in  the  right  half  plane  represents  the  response  to  a  maximum 
command  for  lift.  This  trajectory  represents  the  minimum  time  path  to  a  soft 
contact  with  the  ground.  Crossing  this  boundary  to  the  right  means  that  there  is 
no  action  the  pilot  can  take  to  avoid  a  collision  with  the  ground.  The  dotted  lines 
represent  perceptual  or  feedback  constraints.  These  constraints  are 
approximated  as  a  Weber  fraction  of  the  altitude  and  velocity.  That  is,  the  pilot's 
ability  to  discriminate  a  change  in  altitude  or  velocity  depends  on  the  base  value. 
For  example,  at  higher  altitudes  the  pilot  will  not  be  able  to  resolve  small 
changes  in  altitude  that  would  be  easily  discriminable  at  lower  altitudes.  Figure 
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3  is  a  hypothetical  approximation  to  the  constraints  that  might  set  the  boundaries 
for  safe  low  altitude  flight.  Within  our  framework,  mapping  out  the  actual 
constraints  in  a  dynamic  workspace  is  an  important  step  to  understanding  situation 
awareness  and  adaptation. 

In  the  following  sections  of  the  report,  three  task  spaces  will  be  described. 
The  first  task  space  is  the  control  of  locomotion.  This  is  a  work  that  we  have 
been  doing  in  collaboration  with  Dr.  Rik  Warren  at  the  Armstrong  Laboratory. 
For  the  past  few  years  we  have  been  studying  the  active  psychophysics  of  altitude 
perception  (Flach,  1990;  Flach  &  Warren,  1995a).  This  research  has  demonstrated 
that  the  ability  to  control  altitude  is  a  joint  function  of  the  type  of  texture  (splay 
angle  or  depression  angle)  and  the  nature  of  the  locomotion  (e.g.,  hover  in  a 
rotary  wing  vehicle  versus  high  velocity  forward  motion  as  in  a  fixed  wing 
vehicle).  More  recently,  this  research  has  focused  on  collision  avoidance.  In 
particular,  we  have  been  manipulating  flight  dynamics  and  have  been  testing  the 
extent  to  which  these  action  constraints  are  reflected  in  pilots'  control  strategies. 

The  second  line  of  research  employed  a  dynamic  occlusion  paradigm  to 
compare  performance  of  active  and  passive  observers.  The  dynamic  occlusion 
paradigm  has  been  chosen  as  a  vehicle  for  unconfounding  the  quality  of 
information  available  from  the  mode  (active  versus  passive)  of  the  observer. 
Results  from  a  series  of  experiments  suggests  that  performance  differences  result 
only  when  there  are  information  differences.  When  information  was  equivalent, 
the  mode  of  observer  (active  versus  passive)  had  no  effect. 

A  third  series  of  studies  examine  depth  perception  as  a  a  function  of 
response  mode.  Previous  experiments  had  shown  that  depth  judgments 
depended  on  the  response  mode.  Passive  psychophysical  matching  tasks 
resulted  in  systematic  errors  such  that  the  extent  in  depth  was  increasingly 
underestimated  as  the  distance  from  the  observer  increased.  Such  a  result  is 
consistent  with  the  foreshortening  that  result  due  to  perspective.  However,  blind 
or  ballistic  walking  responses  have  shown  that  subjects  can  accurately  walk  to 
targets  with  little  impact  of  distance.  A  series  of  experiments  suggested  that  the 
response  differences  were  due  to  the  response  frame.  Subjects  were  generally 
accurate  when  creating  distances  relative  to  their  own  position  (egocentric 
frame).  Subjects  tended  to  make  errors  only  when  constructing  distances  relative 
to  arbitrary  standards  in  the  environment  (exocentric  frame). 

The  final  task  space  is  the  domain  of  laparoscopic  surgery.  For  the  past 
year  we  have  been  collaborating  with  Air  Force  surgeon  (Maj.  Daniel  McKellar , 
MD  at  the  WPAFB  Hospital)  to  study  the  decision  and  perceptual  motor 
demands  of  laparoscopic  surgery.  This  work  has  included  field  observations 
where  we  have  observed  during  live  surgeries,  participant  observation  where 
two  students  and  I  have  been  given  4  hours  of  hands  on  training  in  laparoscopic 
surgery  at  the  Ethicon  training  facility,  and  extensive  interviews  with  surgeons. 
We  have  also  developed  a  low  fidelity  simulator  for  studying  perceptual-motor 
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coordination  with  video  images  and  laparoscopic  instruments.  This  domain 
provides  an  important  test  of  our  ability  to  generalize  from  our  basic  research  on 
perceptual-motor  coordination  to  problems  of  situation  awareness  in  an  actual 
workspace. 


2.0  The  Control  of  Locomotion 

Detailed  reports  of  our  studies  on  altitude  control  are  contained  in  two 
Masters  Theses  (Kelly,  1993;  Gamess,  1995)  and  in  a  paper  that  has  been  excepted 
for  publication,  pending  revisions  in  the  Journal  of  Experimental  Psychology: 

Human  Perception  &  Performance  (Hach,  Warren,  Garness,  Stanard,  &  Kelly, 

Under  review).  The  first  part  of  this  section  will  summarize  the  results  of  this 
series  of  studies.  The  second  part  of  this  section  will  report  on  preliminary  work 
directed  at  control  of  collisions.  The  third  subsection  will  describe  a  prototype 
for  a  primary  flight  display  that  we  are  developing.  This  display  utilizes  some  of 
the  geometrical  invariants  that  we  have  studied  in  our  research  on  the  control  of 
locomotion. 

2.1  Altitude  Control 

Gibson,  Olum,  and  Rosenblatt  (1955)  presented  one  of  the  first 
mathematical  descriptions  of  the  optical  flow  field  that  results  from  locomotion 
through  a  textured  environment  (in  particular,  landing  an  aircraft).  From  their 
analysis  they  concluded  that  "the  motion  perspective  of  a  surface  like  the  earth, 
or  a  floor  or  wall,  carries  information  about  the  direction  of  one's  locomotion  (the 
angle  of  approach  to  the  surface)  as  well  as  a  great  deal  of  information  about  the 
surface  itself1  (p.  381).  Gibson  et  al.  identified  two  distinct  characteristics  of  flow 
in  the  visual  field  —  pattern  and  amount.  The  radial  pattern  of  flow  was 
identified  as  a  primary  source  of  information  for  the  direction  of  motion  relative 
to  a  surface.  The  gradients  of  amount'  of  flow  were  identified  as  a  cue  for  the 
perception  of  distance  to  the  surface.  The  analysis  of  the  gradients  of  amount  of 
flow  led  to  the  following  claims  about  the  information  available  to  the  observer 
(O):  "  . " 

O's  linear  velocity  (ground  speed)  is  represented  in  the  optical  flow- 
pattern.  Subjective  velocity  is  proportional  to  the  overall  velocity  of  the 
whole  pattern,  or  to  the  velocity  of  any  part  of  it,  or  to  its  maximal 
velocity.  The  perpendicular  distance  from  O  to  the  surface  (altitude)  is 
also  represented  in  the  optical  flow  pattern,  and  so  is  distance  to  the 
surface  on  the  line  of  locomotion  in  the  case  of  a  landing  glide.  Both  are 
inversely  proportional  to  its  velocity.  Ground-speed  and  altitude  are  not, 
however,  independently  determined  by  the  optical  information.  A  more 
rapid  flow-pattern  may  indicate  either  an  increase  in  speed  or  a  decrease  in 
altitude.  Length  of  time  before  touching  down,  however,  seems  to  be  given 
by  the  optical  information  in  a  univocal  manner  (p.  382). 
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It  took  twenty  years  until  Gibson  et  al.'s  hypothesis,  that  geometrical 
properties  of  the  flow  field  are  the  basis  for  control  of  locomotion,  was  tested 
empirically.  For  example,  R.  Warren's  (1976)  evaluation  of  observer's  ability  to 
identify  the  direction  of  motion  was  one  of  the  early  empirical  studies  to  test 
Gibson  et  al.'s  hypotheses.  Since  that  time  a  number  of  empirical  investigations 
have  attempted  to  link  human  performance  to  various  geometrical  properties  of 
flow  fields  (e.g..  Cutting,  1986;  Cutting,  Springer,  Braren,  &  Johnson,  1992;  Larish 
&  Flach,  1990;  Owen  &  R.  Warren,  1986;  Owen,  R.  Warren,  Jensen,  Mangold,  & 
Hettinger,  1981;  Royden,  Banks,  &  Crowell,  1992;  W.H.  Warren  &  Hannon,  1988; 
1990;  W.H.  Warren  Mestre,  Blackwell,  &  Morris,  1991;  W.H.  Warren,  Morris,  & 
Kalish,  1988). 

The  studies  presented  here  continue  this  program  to  search  for  empirical 
links  between  geometric  properties  of  flow  fields  and  the  control  of  locomotion. 

In  particular,  the  focus  is  on  two  geometrical  properties  of  flow  fields  —  splay 
angle  and  depression  angle — and  the  role  that  these  properties  might  play  in  the 
regulation  of  altitude.  In  the  following  sections,  we  first  define  splay  angle  and 
depression  angle.  Then  we  review  the  previous  empirical  work.  Finally,  we  will 
report  a  series  of  empirical  studies  to  test  a  hypothesis  suggested  by  Hach, 

Hagen,  and  Larish  (1991)  that  has  attempted  to  account  for  apparently  disparate 
findings  in  the  literature. 

2.1.1  Geometric  Analysis 

The  first  step  in  an  analysis  of  the  geometric  properties  of  a  flow  field  is  to 
identify  the  surface  texture  elements  that  "carry"  the  flow.  For  example,  Gibson, 
et  al.'s  (1955)  analysis  assumes  points  as  the  texture  elements.  Denton's  (1980) 
research  on  judgment  of  driving  speed  (edge  rate)  is  an  example  where  the 
significant  texture  elements  were  edges.  In  the  present  analysis,  the  texture 
elements  will  be  edges.  Of  course,  there  is  a  mapping  from  points  to  edges  and 
from  edges  to  points.  In  fact,  it  may  well  be  that  the  distinction  between  points 
and  edges  has  far  greater  implications  for  the  geometrical  analysis,  than  for 
perception.  However,  some  aspects  of  flow  are  more  easily  visualized  and 
modeled  as  properties  of  points,  while  others  are  more  easily  visualized  as 
properties  of  edges.  Splay  and  depression  angle  are  most  easily  visualized  as 
properties  of  edges.  Splay  angle  is  a  property  of  edges  parallel  to  the  direction  of 
motion.  Depression  angle  is  a  property  of  edges  perpendicular  to  the  direction  of 
motion. 

Splay  Angle.  Optical  splay  angle  was  identified  as  a  source  of  information  for 
altitude  by  Warren  (1982).  Warren  cites  Biggs  (1966),  who  noted  that  when  an 
observer  maintains  a  constant  distance  to  an  edge  on  the  ground  plane  (e.g.,  the 
curb  of  the  road),  despite  shifting  optical  positions  of  the  individual  points 
composing  the  edge,  the  optical  position  of  the  edge  was  invariant.  For  an  edge 
parallel  to  the  direction  of  motion,  the  invariant  optical  position  can  be  defined  in 
terms  of  the  angle  at  the  vanishing  point  formed  by  the  edge  and  a  reference  line 
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perpendicular  to  the  horizon  along  the  ground  trace  of  forward  motion,  as 
shown  in  Figure  4.  This  angle  is  defined  by  the  equation: 

S=,an-g) 

where  S  is  the  splay  angle,  Yg  is  the  lateral  displacement  of  the  line  from  the 
perpendicular,  and  z  is  the  altitude  (eyeheight)  of  the  observer.  The  equation 
describes  the  projection  of  the  ground  texture  onto  the  frontal  plane  for  an 
observer  moving  parallel  to  the  ground.  For  rectilinear  motion  over  a  flat 
ground  plane,  splay  angle  is  constant  when  altitude  is  constant. 


The  rate  of  change  in  splay  angle  with  respect  to  change  in  observer 
position  is  specified  by  the  following  equation  (dotted  variables  are  used  to 
specify  temporal  derivatives): 
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The  first  term  [^--Jcos5sin5]  indexes  change  in  splay  as  a  function  of  changes 
in  altitude  [  z]-  The  negative  sign  indicates  that  as  altitude  decreases,  splay  angle 


z  * 

increases,  and  vice  versa.  The  term  [ — ]  specifies  fractional  change  in  altitude  or 

z 

change  in  altitude  scaled  in  eyeheights.  This  term  indicates  that  the  relation 
between  change  in  altitude  and  change  in  splay  angle  depends  on  the  initial 
altitude.  At  high  altitudes  (large  z),  any  given  change  in  altitude  would  result  in 
a  smaller  change  in  splay  angle  than  when  initial  altitude  was  lower.  As  noted 
by  Warren  (1988),  "sensitivity  of  the  display  [optical  splay  rate]  varies  inversely 
with  altitude:  the  lower  the  altitude,  the  more  change  in  visual  effect  for 
equivalent  altitude  change  commands.  At  very  low  altitudes  this  optical  activity 

£ 

is  dramatic  even  optically  violent"  (p.  A121).  The  [ — ]  term  is  independent  of 

z 

optical  position  of  an  edge.  It  scales  the  rate  of  change  for  every  edge  in  the  field 
of  view.  For  this  reason,  it  has  been  termed  "global  perspectival  splay  rate" 
(Wolpert,  1987).  The  sine  and  cosine  terms  index  the  dependence  of  splay  rate 
on  optical  position  of  each  edge.  Figure  5  shows  the  change  of  splay  for  a  5  ft 
decrease  in  altitude  from  an  initial  observation  height  of  25  ft.  as  a  function  of  the 
initial  splay  angle.  For  edges  with  0°  splay  angle  (perpendicular  to  horizon  at 

the  expansion  point)  and  +90°  splay  angle  (the  horizon),  the  rate  of  change  will 
be  zero.  From  these  minima,  the  absolute  change  in  splay  angle  for  a  given 

fractional  change  in  altitude  will  increase  to  a  maximum  at  a  splay  angle  of +45°. 


Figured  Anillustral 
altitude  decreases  fro 
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Figure  5.  The  change  of  splay  or  depression  angle  for  a  5  ft  decrease  in  altitude  from 
an  initial  altitude  of  25  ft  as  a  fucntion  of  the  initial  angular  position  is  illustrated  as  a 
sine  function  with  peaks  at  ±45°.  The  change  of  splay  and  depression  angle  as  a 
function  of  a  5  ft  fore-aft  or  lateral  displacement  is  illustrated  as  a  cos2  function  with 
peak  at  0°.  The  full  range  of  initial  splay  anles  is  typically  visible  in  the  frontal  field 

of  view.  However,  alll  initial  depression  angles  are  not  visibel.  For  the  field  of  view 

simulated  in  our  experiments  initial  depression  angles  below  51.87°  were  outside  of 
the  field  of  view. 


The  second  term  in  the 


equation  for  change  in  splay  angle  [ 
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cos2  S  ] 


indexes  change  in  lateral  distance  [  Yg]  from  the  observer  to  the  edge  such  as 
might  result  from  a  lateral  movement  of  the  observer.  For  straight  ahead 
forward  motion  there  is  no  change  in  lateral  distance  and  this  term  has  no  impact 
on  the  optical  splay  angle.  For  this  reason,  this  term  has  not  typically  been 
included  in  analyses  of  splay.  However,  lateral  displacements  have  sometimes 
been  included  in  the  events  that  have  been  simulated  to  study  altitude  control. 
Thus,  it  is  important  to  understand  the  effects  from  this  term.  The  first  half  of  the 


Y 

term  [— *■]  specifies  lateral  displacement  rate  scaled  in  eyeheights.  Changes  in 

lateral  distance  result  in  proportional  changes  in  splay  angle.  The  second  half  of 
the  term  [  cos  5]  indicates  how  change  in  splay  angle  varies  as  a  function  of  the 
optical  position  of  a  particular  line  element.  Figure  5  illustrates  the  change  in 
splay  angle  as  a  function  of  initial  position  for  a  lateral  change  of  -5  ft.  at  an 
altitude  of  25  ft.  As  can  be  seen  in  Figure  5,  change  in  splay  decreases  from  a 

maximum  for  the  texture  line  directly  below  the  observer  (S=0°)  to  a  minimum 

at  the  horizon  (S=+90°).  It  is  important  to  note  that  whereas  changes  in  altitude 
have  symmetrical  effects  on  edges  spaced  equal  distances  to  each  side  of  the 
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observer,  lateral  motions  cause  a  reduction  in  splay  angle  for  edges  in  the 
direction  of  the  lateral  motion  (negative  splay  angles  become  less  negative  for 
movement  in  a  negative  (left)  direction)  and  an  increase  in  splay  angle  for  edges 
in  the  opposite  direction  from  the  lateral  motion  (positive  splay  angles  become 
more  positive).  Thus,  changes  in  altitude  result  in  changes  in  splay  angle  that  are 
symmetric  around  the  motion  path,  whereas  changes  in  lateral  position  have 
asymmetric  effects. 

Depression  angle.  Optical  depression  angle  provides  yet  another  potential  source 
of  information  for  changing  altitude.  Optical  depression  angle  [  <5  ]  has  been 
defined  as  the  angular  position  below  the  horizon  of  an  edge  perpendicular  to 
the  direction  of  motion  (Flach,  Hagen,  &  Larish,  1992).  However,  to  make  the 
angles  comparable  to  those  used  for  splay  angle,  we  use  the  convention  of 
measuring  optical  depression  angle  from  the  point  directly  below  the  observer  as 
illustrated  in  Figure  6.  The  benefit  of  this  convention  is  that  splay  and 
depression  angle  are  both  referenced  to  the  observer,  whereas  with  the  older 
convention  splay  angle  was  indexed  to  the  observer  and  depression  angle  was 
indexed  to  the  horizon.  This  angle  can  be  expressed  as  a  function  of  altitude  (z) 
and  the  principal  distance  on  the  ground  from  the  observer  to  the  texture 
element  [xg]: 

For  rectilinear  motion  over  a  flat  ground  plane,  the  rate  of  change  of  the 
optical  depression  angle  will  be: 

5  =  -^-jcos<5sin5  +  -^jcos2<5 

The  first  term  ^cos  <5  sin  <5  ]  shows  the  contribution  of  changes  in  the 

observer's  altitude  on  the  optical  depression  angle.  The  relation  between 
depression  angle  and  altitude  is  qualitatively  identical  to  the  relation  between 
splay  angle  and  altitude.  As  with  splay  angle,  the  rate  of  change  in  depression 
angle  scales  with  fractional  changes  in  altitude.  Also,  as  with  splay  angle,  the 
rate  of  change  of  depression  angle  will  depend  on  the  initial  optical  position  of  a 
texture  element.  Rate  of  change  of  depression  angle  will  be  zero  at  depression 

angles  of  0°  (directly  below  the  observer)  and  90°  (the  frontal  horizon)  and  will 

be  maximum  at  a  depression  angle  of  45°-  This  function  is  identical  to  the 
function  for  splay  shown  in  Figure  5.  However,  splay  angles  over  the  range  from 

-90°  to  +90°  are  all  visible  in  the  frontal  field  of  view,  for  depression  angle  only 

0°  to  +90°  is  visible  in  the  frontal  field  of  view  (from  0  to  -90°  is  behind  the 
observer).  In  many  practical  situations  an  even  smaller  range  of  depression 
angles  will  be  available  due  to  limits  in  the  frontal  field  of  view  (e.g.,  occlusion 
due  to  the  bottom  edge  of  a  display  or  window).  The  solid  segment  in  Figure  5 
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shows  the  range  of  depression  angles  visible  for  the  viewing  conditions  used  in 
our  studies. 


Figure  6.  Texture  lines  perpendicular  to  the  forward  path  of  travel  provide 
information  for  change  in  altitude  in  terms  of  depression  angle.  As  the  observer 
moves  from  a  high  altitude  (Zj)  to  a  lower  altitude  (Z2)  the  depression  angle  becomes 
larger  and  the  texture  lines  move  up  in  the  field  of  view  toward  the  horizon. 

The  second  term  in  the  equation  for  rate  of  change  of  depression  angle 
[  (rf  ]cos2  ^  iridexes  changes  in  depression  angle  as  a  result  of  forward  motion 
of  the  observer.  In  the  first  part  of  this  term,  xg  is  proportional  to  the  speed  of 

the  observer.  The  term  [  f^-1]  is  speed  scaled  in  eyeheights.  This  term  has  been 
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identified  as  global  optical  flow  rate  (Warren,  1982).  Thus,  the  rate  at  which 
depression  angle  changes  is  affected  by  both  altitude  and  speed.  The  remaining 
part  of  this  term  [  cos2  5]  accounts  for  changes  in  depression  angle  rate  due  to 
initial  optical  position  of  a  texture  element.  Rate  of  change  of  depression  angle 

due  to  forward  motion  will  be  minimum  at  the  horizon  (90°)  and  will  increase  to 

X 

a  maximum  (i.e.,  exactly  -1-)  at  a  point  directly  below  the  observer  (0°).  This  can 

z 

be  seen  in  Figure  5  where  the  effects  for  a  5  ft  backward  motion  are  shown  for  an 
altitude  of  25  ft.  Thus,  the  lower  the  texture  element  is  in  the  forward  field  of 
view,  the  greater  will  be  the  rate  of  change  in  depression  angle  for  a  given  speed 
of  observer  movement.  Remember,  however,  that  much  of  the  forward  field  is 
occluded  so  that  only  a  subset  of  the  curve  (indicated  by  solid  line)  will  normally 
be  visible. 

Both  splay  angle  and  depression  angle  are  components  of  an  expansion  of 
texture  that  is  associated  with  approach  to  a  surface.  Lee  (1976, 1980)  has  shown 
that  this  expansion  pattern  may  provide  important  information  for  control  of 
locomotion  in  terms  of  tau  or  time-to-contact.  Lee's  analysis  reflects  Gibson  et 
al.'s  (1955)  observation  that  while  altitude  and  speed  are  not  specified 
unambiguously  time  before  touching  down  is  given  in  a  "univocal  manner."  It  is 
important  to  note  that  while  speed  and  altitude  are  ambiguous  for  the  flow  of 
dots  (global  optical  flow)  and  for  the  flow  of  horizontal  edges  (change  in 
depression  angle),  change  of  splay  specifies  change  in  altitude  independent  of 
forward  speed.  This  point  will  be  critical  to  our  hypothesis  for  predicting  an 
interaction  between  texture  (depression/ perpendicular  to  line  of  motion  versus 
splay/ parallel  to  the  line  of  motion  edges)  and  forward  speed  for  the  perception 
and  control  of  altitude. 

2.1.2  Human  Performance 

Wolpert,  Owen,  and  Warren  (1983)  compared  observers'  ability  to  detect 
loss  in  altitude  in  a  simulation  of  flight  with  constant  forward  speed  using  three 
types  of  texture,  as  shown  in  Figure  7:  splay  (parallel,  vertical,  or  meridian) 
texture,  depression  (perpendicular,  horizontal,  or  lateral)  texture,  and  grid 
(square  or  checkerboard)  texture.  Splay  texture  was  chosen  to  isolate  the 
information  available  from  optical  splay,  and  depression  texture  was  chosen  to 
isolate  the  information  available  from  global  optical  density.  The  results 
indicated  that  observers  were  best  able  to  detect  loss  in  altitude  with  splay 
texture.  Performance  was  nominally  worse  with  grid  texture  and  was 
significantly  worse  with  depression  texture.  A  number  of  similar  studies  are 
summarized  by  Wolpert  (1983, 1987;  Wolpert  &  Owen,  1985).  Wolpert  (1987) 
notes  that  in  these  studies  "loss  of  altitude  scaled  in  eyeheights  proved  to  be  the 
functional  variable,  performance  improving  over  increasing  levels  of  that 
variable.  In  contrast,  ground-unit-scaled  loss  in  altitude  showed  a  minimal  effect 
over  the  different  levels"  (p.  24).  Because  the  rate  of  change  of  optical  splay  is 
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directly  related  to  change  of  altitude  scaled  in  eyeheights  [ — ],  whereas  optical 

z 

density  is  related  to  change  in  altitude  scaled  in  ground  units,  splay  was 
nominated  as  the  effective  source  of  information  for  judging  change  in  altitude. 
At  that  time,  no  analysis  of  depression  angle  had  been  made.  This  conclusion 
must  be  reconsidered  in  the  light  of  the  analysis  of  change  in  depression  angle 
which  shows  that  change  of  visual  angle  in  the  depression  texture  also  scales 
with  fractional  change  in  altitude. 


GRID 


SPLAY 


DEPRESS  I CN 


Figure  7.  Three  types  of  ground  textures  that  have  been  used  to  isolate  the 
components  of  optic  flow  associated  with  change  of  altitude. 

Johnson,  Tsang,  Bennett,  and  Phatak  (1989)  employed  a  strategy  similar  to 
that  used  by  Wolpert  et  al.  (1983)  to  isolate  the  optical  information  available  for 
control  of  altitude.  They  used  three  texture  types,  splay  texture  only,  which 
isolates  optical  splay;  depression  texture  (with  a  single  meridian  line  roadway  to 
indicate  flight  path),  which  was  intended  to  isolate  optical  depression  angle;  and 
grid  texture,  which  contains  both  optical  splay  and  optical  depression 
information.  Unlike  Wolpert  et  al.,  who  measured  performance  in  a  passive 
psychophysical  judgment  task,  Johnson  et  al.  used  an  active  control  task. 

Johnson  et  al.  introduced  disturbances  in  both  the  vertical  and  lateral  axes. 
Subjects  were  to  minimize  the  effects  of  the  vertical  disturbance  using  a  single¬ 
axis  velocity  control.  Subjects'  control  actions  had  no  effect  on  the  lateral 
disturbance.  The  lateral  (side-to-side)  disturbance  was  introduced  to  prevent 
subjects  from  using  local  information  such  as  the  position  of  a  meridian  texture 
line  on  the  bottom  of  the  display  (e.g.,  distance  from  the  corner  of  a  rectangular 
display)  to  control  altitude.  In  apparent  contradiction  to  the  results  of  Wolpert  et 
al.,  Johnson  et  al.  found  superior  performance  (lower  tracking  error)  with  the 
depression  and  grid  textures.  Higher  tracking  error  was  found  for  the  splay 
texture. 


A  second  study  by  Johnson  and  his  colleagues  (Johnson,  Bennett, 
O'Donnell  &  Phatak,  1988)  examined  active  control  of  altitude  in  a  hover  task.  In 
this  task,  Johnson  et  al.  included  disturbances  on  three  axes:  altitude  [z],  lateral 
[TJ  (visible  only  in  splay  texture),  and  fore-aft  [xg]  (visible  only  in  depression 
texture).  Performance  was  examined  for  numerous  texture  types,  four  of  which 
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were  of  particular  interest  for  the  present  discussion:  splay,  depression,  grid,  and 
dot.  The  results  showed  equivalent  performance  (both  in  terms  of  tracking  error 
and  correlated  control  power)  for  depression,  grid,  and  dot  textures. 

Performance  with  the  splay  texture  showed  greater  tracking  error  and  lower 
correlated  control  power.  Again,  this  result  is  in  apparent  contradiction  to  the 
findings  of  Wolpert  et  al.  (1983). 

Two  differences  between  Johnson  et  al.'s  studies  and  the  earlier  work  of 
Wolpert  et  al.  were  the  inclusion  of  disturbances  on  axes  other  than  the  altitude 
axis  and  the  use  of  an  active  control  task.  Wolpert  (1988)  employed  an  active 
altitude  regulation  task  with  disturbances  in  altitude  and  roll  (subjects  only 
controlled  altitude).  Note  that  a  roll  disturbance  affects  the  optical  activity  of 
botii  parallel  and  perpendicular  texture,  but  not  the  angular  relations  of  splay 
and  depression  angle.  Wolpert  found  that  "altitude  was  better  maintained  over 
parallel  [splay]  texture  than  over  square  [grid]  or  perpendicular  [depression] 

texture  (p.  17).  Wolpert  found  that  whether  or  not  the  roll  disturbance  was 

included  had  no  effect  on  performance. 

Flach,  Hagen,  and  Larish  (1992)  also  measured  performance  in  an  active 
control  task  with  disturbances  similar  to  those  used  by  Johnson  et  al.  (1988), 
except  that  whereas  Johnson  used  a  hover  task,  Flach  et  al.  used  a  task  with' 
forward  velocity  so  that  the  fore-aft  disturbance,  implemented  as  a  variable 
headwind,  affected  forward  velocity,  not  position.  Flach  et  al.'s  results  were 
consistent  with  Wolpert's.  Performance  was  best  with  splay  and  grid  textures  — 
oth  of  which  contain  splay  information.  Performance  was  poor  in  the 
depression  texture  conditions  contrary  to  Johnson  et  al.'s  results. 


Wolpert  (1988)  also  included  optical  flow  rate  as  a  variable  in  his  study 
He  found  a  performance  decrement  for  increasing  levels  of  global  optical  flow 
rate.  This  is  consistent  with  previous  research  by  Wolpert  and  Owen  (1985). 
They  used  global  optical  flow  rates  corresponding  to  walking  speed  (1 
eyeheight/s)  and  very  low  flight  (.25  &  .5  eyeheights/s)  and  found  that  detection 
of  descent  over  square  texture  deteriorated  with  increasing  global  optical  flow 
rates.  This  is  interesting  in  light  of  the  optical  analysis  presented  earlier.  Optical 
splay  angle  is  independent  of  global  optical  flow  rate.  However,  optical 
depression  angle  is  dependent  on  global  optical  flow  rate.  Global  optical  flow 

rate  ^  ]  changes  as  a  function  of  altitude  (z).  However,  changes  in  global 

optical  flow  are  not  specific  to  altitude.  Global  optical  flow  rate  is  directly 
proportional  to  forward  velocity  [ig  and  inversely  proportional  to  altitude.  This 
had  been  previously  noted  in  the  optical  analysis  of  Gibson  et  al. 

(1955). 


It  is  interesting  to  note  that  the  global  optical  flow  rates 
Wolpert  (1988)  and  Flach  et  al.  (1992)  were  all  greater  than  .25 


examined  by 
eyeheights/s. 
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However,  the  optical  flow  rates  examined  by  Johnson  et  al.  (1988,1989)  ranged 
from  0  for  the  hover  task  to  .25  eyeheights/s.  Thus,  in  the  Johnson  et  al.  studies 
the  optical  flow  rates  were  lower  than  in  previous  studies.  Also,  in  each  of  the 
studies  discussed  above,  the  texture  which  isolated  the  most  effective  optical 
information  (whether  splay  or  depression  angle)  always  yielded  performance 
that  was  superior  to  (though  not  typically  significantly  superior  to)  the  texture 
that  combined  the  two  sources  of  information  (grid  or  dot  texture).  Wolpert  et  al. 
(1983)  and  Wolpert  (1988)  found  that  performance  was  better  with  splay  texture 
than  with  grid  texture.  Johnson  et  al.  (1988, 1989)  found  that  performance  was 
better  with  depression  texture  than  with  grid  or  dot  textures.  Also,  R.  Warren 
(1988)  found  that  altitude  control  was  superior  with  splay-only  texture  than  with 
splay  plus  superimposed  dot  texture.  Why  does  the  combination  of  multiple 
sources  of  information  result  in  performance  degradation? 

Perhaps,  the  optical  activity  resulting  from  forward  motion  (global  optical 
flow  rate)  makes  it  more  difficult  for  the  observer  to  pick-up  the  the  optical 
activity  that  specifies  changes  in  altitude.  In  splay-only  textures,  global  optical 
flow  rate  is  invisible  so  that  there  should  be  no  interference.  If  the  rate  of 
forward  motion  is  slow  or  altitude  is  high,  then  the  contribution  of  global  optical 
flow  will  be  small  so  that  interference  will  be  small.  But  if  global  optical  flow 
rate  is  high  and  can  be  seen  in  the  display  (perpendicular  texture  elements  or 
dots  are  present  in  the  display),  then  the  "noise"  created  by  this  optical  activity 
may  make  it  difficult  for  the  observer  to  disambiguate  changes  in  altitude  from 
changes  of  fore-aft  position. 

Table  1  shows  optical  activity  as  a  function  of  texture  and  motion.  In  the 
experiments  reviewed  altitude  motion  is  the  "signal"  to  which  observers  should 
be  responding.  Optical  activity  from  fore-aft  or  lateral  motions  is  "noise."  The 
hypothesis,  posed  by  Each,  et  al.  (1992),  suggests  that  differential  signal-to-noise 
ratios  across  the  experiments  caused  the  variations  in  performance  observed. 

This  hypothesis  is  tested  in  the  following  experiments. 

2.1. 3  Methodology 

The  subjects'  task  was  to  maintain  a  constant  altitude  in  the  presence  of 
wind  gusts  disturbances.  Disturbances  affected  motion  on  three  axes  —  altitude, 
fore-aft,  and  lateral  (side-to-side).  The  subjects  could  control  only  altitude.  The 
control  was  a  first-order  control  where  stick  displacement  commanded  the  rate 
of  change  of  altitude.  Independent  variables  included  the  ground  texture  and 
the  forward  speed.  Four  ground  textures  were  used  splay  only  (vertical  lines), 
depression  angle  only  (horizontal  lines),  block  (both  splay  and  depression  angle), 
and  dot.  Forward  speed  varied  between  zero  (hover  condition)  and  4 
eyeheights/ s.  Dependent  variables  included  RMS  altitude  error  and  the 
correlation  between  control  power  and  altitude  disturbance  power  in  the 
frequency  domain.  Details  of  the  apparatus  and  methodology  are  available  the 
thesis  and  journal  publications  listed  in  the  introduction  to  this  section. 
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Table  1:  Source  of  Optical  Activity 

SIGNAL  NOISE 


2.1.4  Summary  of  Key  Results 

The  pattern  of  results  across  the  experiments  was  generally  consistent 
with  the  signal-to-noise  hypothesis  of  Flach,  et  al.  (1992).  The  hypothesis  was 
that  performance  in  the  altitude  regulation  task  depended  on  the  ratio  of  optical 
activity  due  to  altitude  (signal)  to  the  optical  activity  arising  from  other  sources 
(noise).  Table  1  shows  how  the  four  textures  effect  the  signal  and  noise 
components  of  optic  flow.  The  specific  prediction  derived  from  this  hypothesis 
was  an  interaction  between  texture  and  forward  flow  rate  (i.ev  global  optical 
flow  rate)  such  that  control  will  deteriorate  for  depression  texture  with 
increasing  flow  rates,  but  that  control  would  be  good  and  independent  of  flow 
rate  for  splay  texture.  The  reason  for  this  prediction  is  that  the  forward  flow  rate 
is  visible  with  depression  texture  and  thus  is  an  increasing  source  of  noise  (i.e., 
optical  activity  unrelated  to  changes  in  altitude)  with  increasing  forward  speeds. 
Forward  flow  is  invisible  for  splay  texture,  thus  no  increase  in  noise  results  with 
increased  forward  flow  rate.  Such  interactions  were  found  in  all  experiments. 

The  results  of  these  experiments  have  important  implications  for 
examining  the  visual  system  within  the  framework  of  ideal  observer  theory  (See 
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Crowell  &  Banks,  In  press).  The  differential  performance  found  for  splay  and 
depression  textures  seems  to  reflect  differences  in  the  quality  of  the  information 
(signal-to-noise)  ratio  and  not  differential  attunement  or  sensitivity  of  the  visual 
system  to  a  specific  component  of  the  optical  expansion  pattern  (i.e.,  splay  angle 
or  depression  angle).  Thus,  when  information  is  most  nearly  equivalent  for  the 
two  components  (0  forward  flow  and  equivalent  initial  angular  positions  in  the 
field  of  view)  near  equivalent  performance  is  observed.  This  is  most  clear  in 
Figure  8  which  shows  similar  levels  of  RMS  altitude  error  for  the  splay  and 
depression  textures  for  zero  forward  flow  rate  and  in  Figure  9  which  shows 
equivalent  levels  of  correlated  control  power  for  splay  and  depression  textures. 

While  performance  is  equivalent  when  the  signal-to-noise  ratio  is  similar 
for  the  two  textures,  there  are  a  number  of  factors  that  contribute  to  an 
information  bias  in  favor  of  the  splay  texture.  First,  as  shown  in  Figure  5  the 
complete  range  of  splay  angles  is  normally  visible  in  the  field  of  regard,  whereas 
only  a  limited  range  of  depression  angles  is  normally  visible.  Also,  the  range  of 
visible  splay  elements  will  normally  include  the  points  of  maximal  absolute 

optical  change  (+  45°),  whereas  this  will  seldom  be  the  case  with  depression 
texture. 


Texture  Type 


Figure  8.  RMS  altitude  error  as  a  function  of  texture  and  forward  flow  rate.  Note 
that  for  the  hover  condidtion  less  error  is  seen  with  depression  texture,  but  with 
forward  motion  splay  texture  results  in  less  error. 
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Figure  9.  Correlated  control  power  as  a  function  of  texture  type,  disturbance  and 
flow  rate,  note  that  the  correlations  with  the  altitude  disturbance  are  equivalent  for 
the  different  textures  in  the  hover  condition.  However,  there  is  a  higher  correlation 
for  splay  texture  when  there  is  signigicant  forward  motion. 

A  second  factor  that  contributes  to  an  information  bias  in  favor  of  the 
spiay  texture  as  information  for  altitude  change  is  the  ubiquitous  presence  and 
magnitudes  of  forward(or  fore-aft)  motion  associated  with  locomotion.  These 
motions  are  a  source  of  noise  for  depression  texture,  but  are  not  visible  in  splay 
texture.  Whenever,  significant  forward  motion  is  present  there  is  a  decline  in  the 
ability  to  regulate  altitude  with  depression  texture.  No  such  decline  is  evident 
for  splay  texture. 

A  final  factor  that  contributes  to  an  information  bias  in  favor  of  splay 
texture  is  a  distinction  in  the  symmetry  of  effects  due  to  lateral  motion  and 
altitude  change  on  splay  angle.  Changes  in  altitude  have  symmetric  effects  on 
sp  ay  angle.  That  is  a  change  in  altitude  has  equal  and  opposite  effects  for  splay 
textures  equal  distances  from  the  right  and  left  of  the  forward  motion  path. 
Lateral  motion  has  an  asymmetric  effect  on  splay  angle.  Splay  angle  decreases 
for  texture  elements  in  the  direction  of  the  motion  and  increases  for  texture 
elements  in  the  opposite  direction  of  the  motion.  Similar  differences  in 
symmetry  are  also  present  for  depression  angle,  however,  due  to  limits  in  the 
field  of  view,  these  asymmetries  in  depression  angle  are  generally  not  visible. 

The  difference  in  symmetry  that  is  visible  for  splay  is  information  that  observers 
can  use  to  disambiguate  optical  activity  associated  with  change  in  altitude  from 
optical  activity  associated  with  lateral  motion. 

There  is  an  important  caveat  with  regard  to  the  optimal  observer  frame 
work  and  the  signal-to-noise  hypothesis  used  here.  Generally,  the  "noise"  in  the 
optimal  observer  framework  reflects  resolution  of  the  visual  processing  system 
(e.g.,  different  resolutions  of  the  central  and  peripheral  retina  —  Crowell  and 
anks.  In  press).  However,  in  the  context  of  our  experiments,  the  distinctions 
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between  signal  and  noise  arise  from  the  specific  experimental  task  used.  Optical 
changes  arising  from  forward  or  lateral  motion  are  only  "noise"  relative  to  the 
goal  of  discriminating  changes  in  altitude.  A  strong  case  could  be  made  that  this 
task  is  not  representative  of  natural  locomotion.  In  natural  locomotion,  all 
disturbances  (altitude,  fore-aft,  and  lateral)  are  important  signals.  In  fact,  it 
might  be  further  argued  that  higher  order  relational  properties  of  these 
components  are  the  true  signals  that  guide  locomotion.  For  example,  with  a 
fixed  wing  aircraft  altitude  depends  on  air  speed.  Increased  airspeed  results  in 
increased  lift  (a  gain  in  altitude),  reduced  airspeed  results  in  reduction  in  lift  (a 
decrease  in  altitude).  Thus,  we  should  be  very  careful  in  generalizing  the  results 
from  the  experiments  reported  here  to  the  general  problem  of  control  of 
locomotion.  This  caveat  applies  equally  well  to  all  the  studies  on  altitude  control 
that  have  been  reviewed  in  the  introduction. 

A  second  caveat,  concerns  the  use  of  the  term  "noise"  to  characterize  the 
optical  effects  due  to  movements  in  axes  other  than  the  vertical  axis.  Two  types 
of  disturbances  were  used.  One  type  of  disturbance  was  a  sum-of  sines 
disturbance.  This  disturbance  was  applied  to  both  the  for-aft  and  lateral  axes.  In 
zero  forward  flow  conditions  this  was  the  only  type  of  "noise"  present.  This  was 
"noise"  in  the  sense  that  the  effects  were  variable  and  not  easily  predicted  by  the 
actor.  The  other  type  of  disturbance  was  that  due  to  the  forward  flow  rate.  This 
was  "noise"  in  the  sense  that  the  optical  effects  were  irrelevant  to  the  problem  of 
controlling  altitude.  However,  the  effects  of  forward  flow  rate  were  constant 
within  a  trial.  It  has  been  suggested  by  Walt  Johnson  (personal  communication) 
that  file  effect  of  flow  rate  might  be  better  understood  in  the  context  of  a  Weber 
fraction.  That  is,  the  ability  to  see  changes  in  the  optical  activity  (i.e.,  due  to 
change  in  altitude)  may  depend  on  the  base  level  of  optical  activity.  In  this 
sense,  global  optical  flow  rate  is  not  noise  in  the  sense  of  being  an  unpredictable 
or  quasi-random  disturbance.  Rather  it  establishes  the  base  line  from  which 
observers  are  sensitive  to  fractional  changes.  Thus,  with  increasing  flow  rate 
proportionally  greater  changes  in  altitude  are  required  to  yield  the  same 
perceptual  effect. 


A  third  caveat  concerns  the  notion  of  "equating  information."  Splay  and 
depression  angles  are  qualitatively  different  kinds  of  angles  as  they  "appear"  in 
the  visual  field  of  view.  Thus,  equating  them  in  terms  of  optical  rates  of  change 
does  not  necessarily  mean  that  they  are  psychophysically  equivalent.  It  will  be 
important  to  test  the  psychophysical  equivalence  of  these  two  sources  of 
information  directly.  A  psychophysical  program  is  needed  to  evaluate  the 
relative  sensitivities  to  the  various  optical  changes. 

To  end  the  discussion,  we  raise  a  final  puzzle  for  future  research.  In  terms 
of  disambiguating  optical  changes  in  depression  angle  due  to  change  in  altitude 
from  those  due  to  change  in  fore-aft  position  (e.g.,  global  optical  flow  rate)  what 
is  the  ideal  field  of  view?  Or  what  is  the  ideal  field  of  regard  —  where  should  the 
observer  look,  out  toward  the  horizon  or  further  down  in  the  field  of  view?  The 
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maximum  change  m  depression  angle  for  a  given  change  in  altitude  will  occur  at 
45°.  Will  performance  improve  when  this  peak  is  included  in  the  field  of  view? 
Early  on  we  would  have  said  yes  to  this  question.  This  thinking  was  part  of  the 
rationale  for  equating  the  textures  (splay  and  depression  angle)  for  peak  rates  of 
change  m  Experiment  2.  However,  discussions  with  Walt  Johnson  have  caused 
us  to  reasses  our  position.  Figure  10  illustrates  the  puzzle.  Figure  10  shows  the 

components  of  optical  change  in  a  forward  field  of  view  including  50°  below  the 
horizon.  The  dotted  line  shows  optical  change  due  to  a  5ft.  change  in  altitude 
from  a  height  of  25  ft.  The  solid  line  shows  absolute  optical  change  due  to 
movement  forward  at  one  eyeheight  per  second  (25  ft/s  at  25  ft.).  Note  that  a 
crossover  occurs  at  an  angle  78.7°  degrees.  Below  this  crossover  changes  due  to 
forward  flow  are  larger  than  changes  due  to  altitude.  Above  this  crossover  the 
opposite  is  true.  Perhaps  relative  change  is  the  critical  factor  determining 
sensitivity.  Thus,  the  observer  should  look  near  the  horizon  where  rates  of 
change  due  to  altitude  are  relatively  large  compared  to  rates  of  change  due  to 
forward  motion.  If  it  is  the  relative  change  that  is  critical,  then  does 
discnminability  depend  on  the  relative  peaks  within  the  field  of  regard  or  on  an 

regard)  funCfa°n  °Ver  13:16  two  curves  (e-g-/  the  average  change  over  the  field  of 
If  relative  change  is  the  critical  variable  for  discriminating  altitude  change 

from  forward  flow,  then  the  crossover  point  seen  in  Figure  10  is  critical  The  6 

crossover  pomt  can  be  computed  by  equating  the  formulas  for  the  two 
components  of  optical  change  and  solving  for  the  angle: 

/  •  \  /  .  x 
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The  crossover  pomt  is  a  tangent  function  of  the  ratio  of  forward  speed  to  rate  of 
change  of  altitude.  In  Figure  10,  the  forward  flow  rate  was  five  times  faster  than 
the  rate  for  change  in  altitude.  If  the  average  change  in  altitude  was  equal  to  the 

average  flow  rate  (i.e.  8  =  tan1 1),  then  the  crossover  point  would  be  at  45°  as 
shown  m  Figure  5  and  the  maximum  advantage  for  altitude  change  relative  to 
forward  motion  occurs  at  about  67.5°.  Important  psychophysical  questions 
remain  about  how  field  of  regard,  field  of  view,  texture,  and  task  dynamics 
interact  to  determine  optimal  strategies  for  extracting  control  relevant 
information  from  optic  flow. 
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Figure  10.  The  change  in  depression  angle  as  a  function  of  optical  position  for  two 
movements.  The  dotted  line  shos  optical  change  due  to  a  5  ft  change  in  altitude  from 
an  altitude  of  25  ft.  The  solid  line  shows  optical  change  resulting  from  a  25  ft  change 

(i.e.,  one  eyehdght  in  fore-aft  position. 

In  conclusion,  the  performance  differences  that  have  in  the  past  been 
attributed  to  different  textures  (splay  or  depression  texture)  seem  to  result  from 
differential  rates  of  optical  change  specific  to  altitude  change,  relative  to  the  rates 
of  optical  change  resulting  from  other  motions.  The  visual  system  does  not 
appear  to  be  differentially  tuned  to  one  component  of  the  expansion  pattern 
(splay  angle)  or  the  other  (depression  angle).  There  appear  to  be  a  number  of 
important  factors  that  create  a  general  information  bias  in  favor  of  splay  angle  as 
a  more  robust  and  salient  source  of  information  for  specifying  altitude  change. 
However,  many  psychophysical  questions  remain  to  be  addressed  before  we  can 
fully  specify  the  limiting  factors  for  extracting  information  from  optic  flow  and 
the  implications  for  the  control  of  locomotion  in  general  and  the  control  of  low 
altitude  flight  in  particular. 


2.2  Control  of  Collision 

Recently,  we  have  turned  our  attention  to  the  problem  of  collision 
avoidance.  There  are  several  reasons  for  this.  First,  we  believe  that  altitude 
control  is  best  seen  as  a  special  case  of  collision  avoidance  (avoiding  collision 
with  the  ground).  Second,  the  problem  of  collision  avoidance  brings  in  the 
problem  of  control  dynamics.  The  dynamics  of  stopping  and  maneuvering  are 
critical  in  determining  how  closely  obstacles  can  be  approached  without 
collisions.  For  example,  safe  following  distance  for  a  car  depends  on  the 
handling  and  braking  qualities.  A  loaded  truck  requires  greater  braking 
distances  than  an  empty  truck.  Thus,  studying  the  control  of  collisions  allows  us 
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to  focus  on  pilots'  abilities  to  perceive  the  boundaries  set  by  the  vehicle  dynamics 
and  the  interaction  of  these  action  constraints  with  fixed  constraints  such  as  the 
ground. 

In  a  pilot  study,  we  varied  the  dynamic  constraints  in  a  terrain  following 
task  as  shown  in  Figure  11.  At  the  start  of  the  task  the  vehicle  was  moving 
toward  a  cliff  at  a  particular  velocity.  The  subject  had  a  discrete  (button  switch) 
control  that  was  used  to  initiate  ascent.  The  subjects'  task  was  to  initiate  ascent  at 
the  last  possible  moment  so  that  the  vehicle  just  passed  over  the  edge  of  the  cliff, 
narrowly  avoiding  collision.  The  two  critical  independent  variables  were  the 
ascent  dynamics  and  the  approach  velocity.  For  half  the  subjects  (Constant 
Ascent  Group)  the  climb  rate  was  constant,  independent  of  forward  velocity.  To 
do  the  task  successfully,  this  group  would  have  to  initiate  ascent  farther  from  the 
cliff  face  when  moving  at  high  velocities  than  when  moving  at  slower  velocities. 
For  the  second  half  of  subjects  (Proportional  Ascent  Group)  the  climb  rate  was 
proportional  to  velocity.  That  is,  at  higher  velocities  the  vehicle  would  climb  at  a 
higher  rate.  The  consequence  of  this  dynamic  was  that  the  optimum  position  for 
initiating  the  ascent  was  independent  of  velocity.  The  subject  could  initiate  climb 
at  a  fixed  distance  from  the  cliff  independent  of  the  forward  velocity.  Figure  5 
shows  the  boundaries  set  by  these  two  dynamics.  For  the  Constant  Ascent 
Group  the  optimal  distance  from  the  cliff  increases  with  increasing  velocity.  For 
the  Proportional  Ascent  Group  the  optimal  distance  is  constant. 

Figure  12  shows  data  from  the  final  block  of  training.  Note  that  this  figure 
takes  the  form  of  a  state  space  diagram.  The  patterns  of  data  within  this  state 
space  suggest  that  the  subjects  are  sensitive  to  the  different  action  boundaries  for 
the  two  dynamics.  The  Constant  Ascent  Group  began  ascent  at  distances  that 
increased  with  increasing  velocity.  The  slope  of  this  function  is  very  similar  to 
that  of  the  optimum  boundary.  The  Proportional  Ascent  Group  also  began 
ascent  at  farther  distances  with  increased  velocities.  However,  the  slope  of  this 
function  is  much  shallower  than  for  the  Constant  Ascent  Group  —  approaching 
the  zero  slope  of  the  optimal  boundary. 
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PROPORTIONAL  ASCENT  RATE  DYNAMIC 


Figure  11.  These  diagrams  illustrate  the  consequences  of  the  two  different 
dynamic  constraints.  For  the  constant  ascent  rate  dynamic,  ascent  rate  is 
constant.  Thus,  to  just  clear  the  diff  the  pilot  must  initiate  the  climb  farther  from 
the  diff  when  the  forward  velodty  is  greater  (a)  than  when  it  is  lower  (b).  For 
the  proportional  ascent  rate  dynamic,  ascent  rate  is  proportional  to  forward 
velodty.  The  result  of  this  is  that  the  pilot  must  initiate  ascent  at  the  same 
position,  independent  of  whether  forward  vdodty  is  high  (c)  or  low  (d). 


Figure  12.  The  solid  line  represents  optimal  performance  for  the  constant  ascent 
rate  dynamic.  The  dotted  line  represents  optimal  performance  for  the 
proportional  ascent  rate  dynamic.  The  filled  drdes  show  actual  obtained 
performance  for  subjects  trained  with  the  constant  ascent  rate  dynamic.  The 
open  drdes  show  actual  performance  for  subjects  trained  in  the  proportional 
ascent  rate  dynamic. 
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The  pilot  data  show  that  subjects  were  capable  of  detecting  and  adjusting 
for  boundaries  defined  by  their  vehicle  dynamics.  However,  the  data  do  not  let 
us  infer  what  structures  in  the  optical  flow  field  specify  the  dynamic  boundaries. 
The  focus  of  this  portion  of  the  research  program  will  be  to  evaluate  the 
interactions  (interrelations)  between  dynamic  constraints  and  feedback 
constraints  in  terms  of  optical  structure  in  flow  fields.  The  general  strategy  will 
be  to  design  experiments  where  changes  in  dynamics  are  crossed  with  changes  in 
the  optical  structure  of  the  flow  field.  Important  optical  structures  that  will  be 
manipulated  will  include  edge  rate,  global  optical  flow  rate,  the  rate  of  optical 
expansion  (tau),  and  local  size  and  distance  cues.  For  example,  tau,  which  is 
derived  from  the  optical  expansion  rate  of  an  approaching  object,  specifies  an 
impending  collision  in  terms  of  time-to-contact  (e.g.,  Kaiser  &  Mowafy,  1993; 

Lee,  1980;  Tresilian,  1994).  Tau  takes  into  account  both  speed  and  distance.  Tau 
could  be  an  important  source  of  information  for  the  constant  ascent  dynamic 
conditions.  However,  because  the  appropriate  response  for  the  proportional 
ascent  dynamic  is  independent  of  forward  speed,  dependence  on  tau  might  lead 
to  inappropriate  control  strategies.  An  experiment,  will  cross  information  for  tau 
(e.g.,  texture  on  the  cliff  face)  with  other  sources  of  information  for  approach 
speed  (e.g.,  edge  rate  which  will  vary  with  spacing  of  ground  texture  and  global 
optical  flow  which  will  vary  with  the  altitude  of  the  approach  trajectory).  These 
information  variables  will  be  crossed  with  the  control  dynamics.  The  factorial 
crossing  of  the  approach  dynamic  with  information  for  time-to-contact,  and 
information  for  the  speed  of  self-motion  will  help  us  to  learn  the  extent  to  which 
the  control  of  collision  is  based  on  tau  alone  or  to  what  extent  it  is  influenced  by 
alternative  sources  of  information  for  self-motion  velocity.  The  dynamic  variable 
allows  us  to  measure  the  reliance  on  tau  both  when  it  is  and  is  not  an  appropriate 
source  of  information.  If  the  human  visual  system  has  specialized  mechanisms 
for  detecting  and  using  tau,  then  tau's  influence  might  be  seen  even  when  the 
information  provided  is  not  optimal  for  the  control  dynamics.  Indeed,  Figure  12 
shows  some  evidence  that  even  with  the  proportional  ascent  rate  dynamics 
(where  tau  is  inappropriate)  subjects  persisted  to  initiate  their  ascent  at  further 
distances  when  forward  speed  was  greater.  Another  way  to  phrase  the  question 
is  —  what  are  the  visual  primitives  that  are  used  in  controlling  collisions?  We 
know  that  depending  on  the  control  dynamics  the  correct  actions  may  depend  on 
both  position  and  velocity.  However,  are  position  and  velocity  independently 
processed  by  the  visual  system  and  then  integrated  in  the  decision  process  or  is 
the  perceptual  primitive  some  higher-order  variable  (such  as  tau)  in  which 
position  and  velocity  are  naturally  integrated?  If  this  last  case  is  true,  then  one 
might  expect  to  see  the  residual  influences  of  velocity  even  when  the  dynamic 
does  not  require  actions  to  vary  with  velocity. 

2.3  Configural  Attitude  Display 

As  a  result  of  our  analysis  of  the  optical  invariants  that  specify  self-motion 
(e.g.,  Flach  &  Warren,  1995).  We  have  been  developing  a  prototype  for  a  primary 
flight  display  that  incorporates  key  elements  of  the  optical  flow  field  within  a 
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single  configural  representation.  This  display  which  is  illustrated  in  Figure  13. 
integrates  information  about  pitch,  roll,  altitude,  air  speed,  and  heading  into  a 
single  perceptual  object.  Pitch  and  roll  use  standard  inside-out  symbology. 
Altitude  is  represented  in  terms  of  splay  lines.  Air  speed  is  represented  as  flow 
lines.  Heading  is  represented  in  terms  of  a  compass  which  frames  the  object. 
Code  for  a  working  prototype  is  being  developed.  We  hope  to  test  this  display 
with  air  force  pilots  in  facilities  at  either  the  Armstrong  or  Wright  Laboratories 
within  the  coming  year. 


N 


Figure  13.  The  configural  atitude  display  integrates  pitch,  roll,  altitude,  air  speed  and 
heading  into  a  single  configural  representation. 
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3.0  Dynamic  Occlusion 

More  complete  descriptions  of  this  research  are  available  in  three  Master 
Theses  conducted  at  Wright  State  University  (Allen,  1993;  Brickman,  1994; 
Hutton,  1993)  a  paper  is  also  currently  under  review  for  publication  in  the 
journal  Human  Factors  (Flach,  Allen,  Bell,  Brickman,  &  Hutton,  Under  review). 

One  potential  problem  with  increased  automation  in  human-machine 
systems  has  been  identified  by  Wickens  (1984)  as  "out-of-the-loop  familiarity." 
That  is,  if  the  operator  is  not  actively  engaged  within  a  control  loop  (for  example, 
when  the  auto  pilot  is  engaged),  then  s/he  will  be  slower  to  detect  errors  and 
slower  to  exert  the  appropriate  corrective  actions  once  errors  are  detected. 
Wickens  cites  numerous  studies  that  have  found  increased  detection  latency  and 
reduced  accuracy,  when  the  operator  is  not  actively  controlling  the  process 
(Ephrath  &  Young,  1981;  Kessel  &  Wickens,  1982;  Wickens  &  Kessel,  1979, 1980). 
Wicken's  observation,  that  there  is  an  advantage  for  the  active  observer 
compared  to  the  passive  observer,  also  is  supported  by  basic  research  in  human 
performance.  This  basic  research  has  demonstrated  improved  performance  for 
active  observers  in  identifying  objects  by  touch  (Gibson,  1962)  and  in  developing 
perceptual  motor  coordination  and  in  adapting  to  perceptual  distortions  (Held, 
1965;  Held  &  Bossom,  1961;  Held  &  Freeman,  1963;  Held  &  Hein,  1958;  1963; 

Held  &  Shank,  1959). 

However,  there  are  also  numerous  studies  that  raise  questions  about  what 
it  means  to  be  "in-the-loop"  or  to  be  "actively  engaged."  Bell  (1989)  reviews 
several  studies  that  show  superior  fault  detection  performance  for  "auto  pilot" 
conditions  (Ephrath  &  Curry,  1977;  Ephrath  &  Young,  1981,  Experiment  2).  Bell 
suggests  that  many  factors  including  task  complexity,  attention,  workload, 
information,  failure  type,  and  task  type  may  interact  with  the  "mode"  of  the 
operator  (in  or  out-of-the-loop  or  active  versus  passive)  to  determine  the  ultimate 
quality  of  performance.  Also,  basic  research  has  raised  questions  about  the 
importance  of  mode  relative  to  other  variables.  Schwartz,  Perey,  and  Azulay 
(1975)  replicated  Gibson's  (1962)  touch  experiment.  They  found  that  when  care 
was  taken  to  match  the  quality  of  stimulation  for  active  and  passive  observers 
the  passive  observers  improved  to  a  level  comparable  to  active  observers.  Walk, 
Shepherd,  and  Miller  (1988)  replicated  Held  and  Hein's  (1963)  study  of  the 
importance  of  movement  produced  stimulation  on  the  development  of  visually 
guided  behavior.  Their  results  suggested  that  when  differences  in  attentional 
engagement  between  active  and  passive  kittens  were  controlled,  the  passive 
kittens  showed  no  evidence  of  retarded  development  as  found  by  Held  and 
Hein.  Welch  (1986)  in  an  extensive  review  of  the  adaptation  literature  concluded 
that  "it  now  seems  clear  that  although  active  perceptual-motor  behavior  is  an 
ideal  condition  for  the  production  of  adaptation  to  perceptual  rearrangement,  it 
is  not  a  necessary  one.  Rather,  a  variety  of  forms  of  information  (e.g.,  error- 
corrective  feedback,  passive  exposure  to  sharply  defined  visual-proprioceptive 
discrepancy)  will  suffice"  (p.  24.37). 
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Upon  reviewing  the  research  on  the  role  of  mode  (active  versus  passive)  in 
human  performance  it  becomes  clear  that  it  is  extremely  difficult  to  separate  the 
effects  of  mode  from  other  variables.  However,  understanding  the  role  of  mode 
has  important  implications  for  applied  and  basic  human  performance  theory. 

For  applications,  it  has  important  implications  for  the  design  and  use  of 
automation  for  vehicular  and  process  control.  For  basic  research,  it  has 
important  implications  for  research  methodologies.  Laboratory  research  on 
perception  often  is  designed  so  that  the  loop  between  perception  and  action  is 
cut.  That  is,  in  order  that  the  experimenters  have  complete  control  over  the 
"stimuli,"  perceptual  tasks  are  designed  so  that  the  stimulation  is  completely 
divorced  from  the  subjects’  responses.  In  these  domains  the  subjects  are 
essentially  passive  observers  who  make  judgments  about  stimuli  that  are 
imposed  by  an  experimenter.  Will  performance  of  these  passive  observers 
generalize  to  situations  outside  the  laboratory  where  the  human  is  more  likely  to 
be  actively  in  control  of  stimulation?  Flach  and  Warren  (Flach,  1990, 1993;  Flach 
&  Warren,  In  press;  Warren  &  McMillan,  1984)  have  recently  argued  that  an 
"active  psychophysics"  is  required  to  extend  and  test  the  generality  of 
conclusions  based  on  passive  laboratory  paradigms. 

3.1  Methodology. 

To  help  further  understand  the  role  of  action  in  perception,  a  series  of 
experiments  was  conducted  using  a  dynamic  occlusion  paradigm  derived  from 
Stappers  (1989).  The  subjects'  task  was  to  identify  a  3-dimensional  wire  frame 
object  based  on  the  occlusion  and  disocclusion  of  point  lights  resulting  from 
motion  of  the  otherwise  invisible  object.  The  key  manipulation  was  the  mode  for 
producing  motion  of  the  object.  Subjects  were  tested  in  yoked  pairs.  One  subject 
(Active)  controlled  the  motion  of  the  object  by  manipulating  a  two-dimensional 
joy  stick  control.  The  second  subject  (Passive)  had  no  control  over  object  motion, 
but  saw  the  pattern  of  motion  produced  by  the  active  subject.  The  dynamic 
occlusion  paradigm  provides  an  opportunity  to  match  subjects  for  information 
and  attentional  demands.  That  is,  both  subjects  saw  identical  visual  displays 
(i.e.,  the  patterns  of  occlusion  and  disocclusion  were  identical).  Also,  the 
identical  displays  provided  a  comparable  attentional  frame  and  the  goal  of 
identifying  the  object  shape  was  equivalent  for  both  subjects.  Using  a  similar 
task  (with  two-dimensional  objects),  Stappers  (1989)  provided  evidence  that 
there  would  be  an  advantage  for  the  active  observer.  Thus,  our  hypothesis  was 
that  the  active  observers  would  be  better  able  to  identify  the  objects  than  the 
passive  observers.  We  hoped  that  the  dynamic  occlusion  task  would  reveal  the 
contribution  of  mode  unconfounded  with  differences  due  to  information  or 
attentional  factors. 

3.2  Results 

When  we  initiated  this  research  program  we  expected  that  there  would  be 
an  advantage  for  the  active  subjects  in  this  task.  This  prediction  was  not 
supported  by  the  data.  Numerous  experiments  under  a  wide  range  of  conditions 
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showed  consistently  that  there  was  little  difference  as  a  function  of  Mode  when 
information  was  equivalent.  Our  experiments  suggest  that  when  attention  and 
information  factors  were  controlled,  there  was  no  advantages  to  being  "in-the- 
loop."  This  is  consistent  with  studies  by  Schwartz,  Perey,  and  Azulay  (1975)  and 
Walk,  Shepherd,  and  Miller  (1988).  In  those  studies,  differences  previously 
attributed  to  mode  (Gibson,  1962;  Held  &  Hein,  1963)  disappeared  when 
attention  and  information  were  equated  across  conditions. 

The  results  from  this  study  are  also  consistent  with  a  recent  review  of 
research  on  adaptation  to  visual  rearrangement  (Welch,  1986).  Held  and  his 
colleagues  (e.g.,  Hein  &  Held,  1962)  had  argued  that  active  bodily  movement 
was  a  necessary  condition  for  adaptation  to  visual  rearrangement.  However 
Welch  cites  numerous  studies  that  have  found  that  "passive  bodily  movement 
does  lead  to  a  certain  amount  of  adaptation,  although  frequently  less  than 
produced  by  active  movement"  (p.  24.10).  Howard  and  Templeton  (1966) 
suggest  that  the  advantages  that  have  been  observed  for  active  subjects  in  many 
adaptation  studies  might  be  due  to  the  salient  information  about  the  visual 
rearrangement  that  is  provided  by  action.  Welch  (1986)  calls  this  the 
"information  hypothesis."  In  other  words,  it  is  the  information  that  is  critical  to 
performance,  not  whether  that  information  arises  from  subject  activity  or  not.  Of 
course,  in  most  situations  activity  is  critical  for  making  information  available  and 
for  directing  the  actor's  attention.  Thus,  it  is  usually  the  case  that  constraints  on 
action  will,  in  effect,  be  constraints  on  information. 

Bell  (1989)  draws  a  similar  conclusion  from  the  conflicting  findings  on  the 
benefits  of  being  "in-the-loop"  for  the  detection  and  recovery  from  failures  in 
human-machine  systems.  Bell's  analysis  suggests  that  being  "in-the-loop"  will 
facilitate  detection  of  failures  only  when  there  is  information  that  is  only 
available  (or  that  is  more  salient)  to  the  pilot  who  is  manually  engaged  with  the 
system  (e.g.,  proprioceptive  feedback). 

Of  course,  great  care  must  be  taken  in  generalizing  from  the  results  of  this 
study.  First,  we  are  generalizing  based  on  a  null  result.  Obviously,  we  cannot 
prove  that  active  and  passive  subjects  are  equal.  However,  the  data  does  suggest 
that  the  effects  of  Mode  are  small  relative  to  individual  differences  and  relative  to 
other  factors  such  as  density  and  practice.  The  significant  effects  of  density  and 
practice  provide  evidence  that  the  failure  to  find  an  effect  of  Mode  was  not  due 
to  simple  ceiling  or  floor  effects. 

A  second  concern  for  generalization  is  the  dynamic  occlusion  task  itself.  It 
is  possible  that  the  task  of  identifying  the  shape  of  an  object  is  not  representative 
of  the  kinds  of  tasks  where  self  generated  action  would  be  necessary.  The 
attributes  of  the  dynamic  occlusion  task  that  make  it  easy  to  equate  active  and 
passive  subjects  in  terms  of  information  and  attention  may  be  exactly  the 
attributes  that  make  action  unnecessary.  For  example,  the  control  of  self  motion 
may  be  very  different  than  shape  identification.  In  controlling  self  motion. 
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control  inputs  to  limbs  or  to  a  vehicle  may  be  critical  for  determining  the 
response  dynamics  of  the  system.  Shape  identification  is  a  purely  kinematic  task, 
there  are  no  dynamics.  However,  this  is  not  necessarily  a  threat  to  the 
information  hypothesis.  Action  (e.g.,  pulsing  your  brakes  to  test  for  changes  in 
the  braking  dynamics  due  to  a  change  in  surface  properties  due  to  snow)  may  be 
the  only  path  to  certain  kinds  of  information.  The  bottom  line  then,  would  be 
consistent  with  Howard  and  Templeton's  (1966)  conclusion  that  action  is  only 
important  to  the  extent  that  it  differentially  affects  access  to  information. 

In  sum,  the  results  from  this  study  add  a  small  grain  of  evidence  in  favor 
of  the  "information  hypothesis."  Information,  not  mode,  is  the  critical  variable 
ultimately  constraining  situation  awareness.  The  dynamic  occlusion  task  is 
designed  so  that  there  is  no  information  advantage  for  the  active  observer  —  as  a 
result  there  is  no  corresponding  performance  advantage.  In  making  decisions 
about  the  use  of  automation  in  complex  systems,  it  appears  that  the  critical  issue 
for  design  is  how  the  automation  constrains  the  operators'  access  to  information. 
If  the  automation  enhances  the  information  available  to  the  operator,  then  the 
human-machine  system  should  show  improved  situation  awareness.  However, 
if,  as  is  generally  the  case,  the  automation  distances  the  operator  from  critical 
information  about  the  system,  then  the  human's  ability  to  detect  failures  and  to 
respond  adaptively  to  those  failures  will  be  compromised. 


4.0  Depth  Perception 

The  problem  of  depth  perception  combines  interesting  aspects  of  both 
contol  of  locomotion  and  of  the  difference  between  active  and  passive  observers. 
Obviously,  control  of  locomotion  involves  making  judgments  about  the  relative 
distances  of  objects  in  the  field  of  view.  Loomis,  DaSilva,  Fujita,  &  Fukusima, 
1992)  showed  that  performance  in  a  distance  judgment  task  varied  as  a  function 
of  the  response  mode.  When  passive,  matching  tasks  were  used,  subjects 
showed  systematic  biases.  These  biases  were  consistent  with  the  foreshortening 
that  results  when  three-dimensions  are  projected  onto  a  two-dimensional  field 
using  linear  perspective.  However,  Loomis  et  al.  and  others  (e.g.,  Thompson 
(1980;  1983)  have  shown  that  subjects  show  little  error  when  blindly  walking  to  a 
target  in  depth.  Thus,  different  performance  has  resulted  for  active  and  passive 
observers.  A  series  of  studies  was  conducted  in  an  attempt  to  better  understand 
the  factors  that  influence  depth  judgments.  For  complete  details  of  the 
methodology  and  results  see  a  Masters  Thesis  published  at  Wright  State 
University  (Robison,  1995). 

4.1  Methodology 

A  series  of  experiments  were  conducted  to  study  depth  judgments.  Two 
response  modes  were  used.  The  first  mode  was  pointing  —  subjects  made  their 
responses  using  a  laser  pointer.  The  second  mode  was  blind  or  ballistic  walking 
—  subjects  walked  to  the  target  with  their  eyes  closed.  Subjects  were  to  match 
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standard  distances  that  were  indicated  by  markers  on  the  ground.  These 
standards  could  be  either  in  the  frontal  plane  or  in  the  depth  plane.  The 
responses  and  standards  could  be  either  egocentric  or  exocentric.  An  egocentric 
distance  was  distance  relative  to  the  subject.  An  exocentric  distance  was  distance 
relative  to  an  arbitrary  marker  on  the  ground. 

4.2  Results 

The  original  hypothesis  was  that  distance  might  be  scaled  in  terms  of  an 
action  metric.  That  is,  that  blind  walking  would  generally  yield  accurate 
judgments.  However,  the  results  showed  that  accurate  performance  can  be 
obtained  for  both  walking  and  pointing  response  modes.  The  critical  variable 
appeared  to  be  the  response  frame.  That  is,  whenever  responses  were  made  in 
terms  of  distance  from  the  observer  (egocentric),  as  opposed  to  distance  between 
arbitary  points  on  the  gound  (exocentric),  accurate  judgments  of  depth  were 
obtained. 

5.0  Minimally  Invasive  Surgery 

Advances  in  video  and  imaging  technology  have  made  it  possible  for 
many  surgeries  to  be  accomplished  using  noninvasive  techniques.  These 
noninvasive  techniques  have  many  benefits  for  patients  (e.g.,  less  damage  to 
healthy  tissue,  less  scarring,  and  shorter  recovery  time).  A  cholecystectomy  (gall 
bladder  removal)  typically  involves  8-10  days  in  the  hospital  and  3  weeks  to  a 
month  for  full  recovery  when  done  using  a  traditional  open  procedure. 

However,  when  performed  with  a  laparoscope,  patients  can  return  home  as  soon 
as  10  hours  following  surgery  and  can  return  to  work  within  a  few  days.  These 
benefits  to  patients,  however,  are  purchased  at  the  cost  of  far  greater  demands 
on  the  surgeon's  perceptual  and  motor  skills.  With  the  laparoscope  the 
perceptual-motor  loop  depends  on  a  two-dimensional  video  image  for  feedback. 
Not  only  is  the  image  2-dimensional,  but  the  normal  eye-hand  axis  is  replaced  by 

the  camera-hand  axis.  Typically,  the  camera  will  be  oriented  approximately  90° 
from  the  normal  hand-eye  axis.  Thus,  a  movement  from  right-to-left  with  regard 
to  the  normal  eye-hand  axis  will  appear  as  a  movement  in  depth  on  the  video 
monitor.  To  make  things  more  difficult,  the  camera  is  not  fixed,  but  is  moving 
under  the  control  of  an  assisting  surgeon.  While  the  mobility  of  the  camera 
allows  the  surgeon  to  look  around  and  get  the  best  perspective  for  each 
manipulation,  the  changing  camera  orientation  creates  a  challenging  and 
interesting  problem  for  perceptual-motor  coordination. 

In  order  to  understand  the  unique  demands  of  laparoscopic  surgery  we 
have  initiated  two  lines  of  research.  The  first  line  of  research  is  a  field  study  to 
identify  the  critical  perceptual,  motor,  and  cognitive  demands  that  shape 
performance  in  surgery.  The  second  line  of  research  is  a  laboratory  based 
program  of  experiments  using  a  low  fidelity  simulation  to  evaluate  the 
development  of  perceptual  motor  skills. 
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5.1  Field  Studies 

Over  the  past  year  we  have  been  collaborating  with  physicians  at  the 
Wright  State  School  of  Medicine  (Dr.  Margaret  Dunn)  and  at  the  Wright- 
Patterson  Air  Force  Base  Hospital  (Dr.  Daniel  McKellar)  to  study  the  challenges 
associated  with  noninvasive  surgery.  These  studies  have  included  observations 
in  surgery,  participant  observation  in  which  two  of  my  students  and  I  received  4 
hours  of  hands  on  training  in  laparoscopic  surgery,  and  interviews  with  local 
surgeons.  We  are  currently  conducting  structured  interviews  with  expert  (senior 
staff  physicians  with)  and  novice  (senior  residents  with  less  surgical  experience) 
surgeons  using  video  from  actual  surgeries.  Dr.  Dan  McKellar  chose  the  videos 
and  helped  us  to  edit  them  down  to  an  appropriate  length  and  format  for  the 
interviews.  The  structured  interviews  are  recorded  on  audio  tape.  Then  they  are 
transcribed  and  coded  using  the  MacSHAPA  software.  (MacSHAPA  is  an 
exploratory  data  analysis  tool  for  analyzing  sequential  data  such  as  verbal 
protocols.  It  was  developed  by  Dr.  Penny  Sanderson  at  the  University  of  Ulinois 
with  support  from  the  Armstrong  Laboratory  at  WPAFB).  At  this  date  twenty 
interviews  have  been  conducted  (10  novices  and  10  experts)  and  the  transcription 
process  is  just  beginning. 

The  transcripts  from  the  interviews  are  being  coded  using  categories 
derived  from  Rasmussen's  work  on  cognitive  systems  engineering  (e.g, 
Rasmussen,  Pejtersen,  &  Goodstein,  1994).  The  coding  system  will  look  at  the 
goal  structures  in  terms  of  Rasmussen's  abstraction  hierarchy,  the  decision 
making  activities  in  terms  of  Rasmussen's  decision  ladder,  and  the  cognitive 
strategies  employed  by  novices  and  experts.  The  primary  objective  of  the  verbal 
protocol  analysis  will  be  to  identify  differences  in  how  experts  and  novices 
approach  decision  making  in  surgery.  One  hypothesis  is  that  experts  will  show 
greater  awareness  of  the  hierarchical  goal  context  as  represented  by  the 
abstraction  hierarchy.  This  should  show  up  in  the  protocols  as  more  references 
to  higher-order  constraints  in  terms  of  references  to  global  goals  and  abstract 
functional  constraints.  In  short,  the  experts  will  have  a  richer  context  for 
articulating  "why"  certain  considerations  are  important.  A  second  hypothesis  is 
that  experts  will  tend  to  rely  on  perceptual,  recognition-based  decision  strategies 
and  will  tend  to  generate  fewer,  but  better  alternatives  than  novices.  This 
hypothesis  is  consistent  with  work  by  Klein  (1989)  who  has  found  that  experts 
depend  heavily  on  perceptual  skills  to  identify  alternative  courses  of  actions. 
These  experts  tend  to  generate  and  evaluate  single  options,  rather  than  to  do 
comparisons  across  several  options.  Experts  decision  making  might  be 
characterized  as  narrow  and  deep,  when  compared  to  novices  whose  reasoning 
processes  tend  to  be  broader  and  more  shallow.  Such  strategies  are  successful 
because  experts  are  able  to  recognize  and  classify  situations  based  on  past 
experiences.  We  believe  that  these  expert-novice  distinctions  may  be  a  critical 
variable  in  understanding  situation-awareness. 
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The  long  term  goal  for  this  project  is  to  develop  training  protocols  to 
accelerate  the  development  of  expertise.  The  nature  of  the  training  protocols  will 
be  contingent  upon  the  results  from  the  protocol  analyses.  The  general  plan  is  to 
integrate  the  video  tapes  used  in  the  knowledge  elicitation  process  with  excerpts 
from  the  audio  transcripts  to  illustrate  the  contextual  scope  and  the  perceptual 
distinctions  that  experts  use  in  making  decisions.  The  resulting  training  protocol 
will  be  evaluated  using  junior  surgical  residents  in  the  Wright  State  Surgical 
Residency  Program. 

5.2  Laboratory  Research 

The  laboratory  research  on  laparoscopic  surgery  employs  a  low  fidelity 
simulation  that  we  have  developed.  The  simulator  illustrated  in  Figure  8 
requires  subjects  to  pick  up  small  plastic  chips  using  an  Endo-Babcock  surgical 
instrument  (this  instrument  is  a  racheting  clamp  used  to  grasp  and  manipulate 
tissue  and  organs  during  laparoscopic  surgeries).  The  chips  are  picked-up  from 
one  container  using  the  left  instrument,  then  the  chip  must  be  transfered  to  the 
right  instrument,  and  then  deposited  in  a  second  container.  To  complete  a  trial  8 
chips  must  be  transfered.  If  a  chip  is  dropped,  then  the  subject  must  pick-up 
another  chip  from  the  container.  The  dependent  measure  is  the  time  to  complete 
the  8  transfers.  An  occluding  screen  prevents  direct  view  of  the  workspace. 
Visual  feedback  is  provided  using  a  video  camera  and  a  standard  video  monitor. 
The  simple  arrangement  makes  it  very  simple  to  manipulate  the  position  of  the 
camera  relative  to  the  subject  and  the  workspace. 

This  simulation  was  designed  to  replicate  some  of  the  perceptual-motor 
demands  of  surgery  while  at  the  same  time  being  simple  enough  so  that 
undergraduates  with  no  surgical  training  can  be  used  as  subjects.  Preliminary 
testing  suggest  that  we  were  successful  on  both  counts.  The  task  has  been  tried 
by  three  different  surgeons  and  a  representative  from  a  medical  instrumentation 
company.  The  consensus  of  these  experts  is  that  the  perceptual-motor  demands 
are  representative  of  the  demands  of  surgery.  We  have  also  conducted  a  pilot 
test  with  a  small  group  of  undergraduates.  These  students  showed  initial 
frustration  with  the  task,  but  were  able  to  master  the  task  over  a  period  of  about 
one  week  of  practice  for  one  hour  per  day.  The  subjects  were  to  pick  up  and 
transfer  8  small  plastic  chips  from  one  container  to  another.  The  transfer 
required  that  the  object  be  picked  up  with  the  left  instrument,  passed  to  the  right 
instrument,  and  then  deposited  in  a  container.  Initial  mean  times  were 
approximately  800  s.  However,  the  subjects  asymptoted  at  about  100  s  by  the 
end  of  the  experiment. 

The  first  experiment  using  this  simulation  will  examine  the  relative 
importance  of  three  relations  within  the  perceptual-motor  loop  —  the  relation 
between  the  hand  and  the  camera,  the  relation  between  the  hand  and  the  work, 
and  the  relation  between  the  camera  and  the  work.  A  transfer  paradigm  will  be 
used  to  assess  the  relative  importance  of  the  links.  Three  groups  of  subjects  will 
be  trained  using  the  schematic  setup  illustrated  in  Figure  14.  The  camera  will  be 
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at  the  left  hand.  They  will  have  to  pick  up  objects  from  the  left  container  and 
transfer  them  to  the  container  on  the  right  hand  side.  Each  group  will  then 
transfer  to  one  of  the  three  arrangements  illustrated  in  Figure  15.  In  the  Transfer 
A  condition,  the  position  of  the  camera  is  moved  (from  the  left  hand  to  the  right 
hand).  With  this  change,  the  relation  between  the  hand  and  the  work  remains 
invariant,  but  the  relations  between  the  hand  and  the  camera  and  between  the 
camera  and  the  work  are  altered.  In  the  Transfer  B  condition,  both  the  camera 
and  the  work  are  moved  so  that  the  relation  between  the  camera  and  the  work  is 
invariant,  but  the  relations  between  the  hand  and  the  camera  and  between  the 
hand  and  the  work  are  altered  (i.e.,  the  chips  must  now  be  picked-up  with  the 
right  hand  and  transfered  to  the  left  before  being  deposited  in  the  left  container). 
Finally,  Transfer  C,  is  designed  to  keep  the  relation  between  the  hand  and 
camera  invariant  while  altering  the  relations  between  hand  and  work  and 
between  work  and  camera.  This  last  condition  is  created  by  reversing  the 
workspace  without  changing  the  position  of  the  camera.  Table  2  summarizes  the 
relations  for  the  three  transfer  condition.  An  equal  sign  (=)  signifies  a  link  that 

remains  invariant  and  a  not-equal  sign  (*)  signifies  a  link  that  is  altered  as  a 
result  of  transfer. 


TABLE  2 


LINK . 

TRANSFER  A 

TRANSFER'! 

Hand/Work 

= 

* 

* 

Hand/  Camera 

* 

* 

Camera /Work 

5* 

= 

The  transfer  conditions  were  designed  so  that  a  different  relation  remains 
invariant  in  each  transfer  condition.  This  will  allow  us  to  assess  the  relative 
importance  of  each  link.  The  assumption  is  that  the  size  of  the  performance 
decrement  upon  transfer  will  be  an  index  of  the  importance  of  the  link.  If  a  link 
is  important,  then  there  should  be  a  large  performance  decrement  when  that  link 
is  altered,  but  little  decrement  when  that  link  is  invariant  between  training  and 
transfer  conditions.  Pilot  testing  showed  a  large  decrement  for  the  Transfer  B 
condition,  but  only  a  small  decrement  for  the  Transfer  A  condition.  These  results 
suggest  that  the  link  between  the  hand  and  work  is  important,  because  there  was 
little  decrement  when  this  link  remained  invariant,  but  a  large  decrement  when 
this  link  was  altered.  No  data  is  available  for  the  Transfer  C  condition. 


Video  Display 
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PERSPECTIVE  VIEW 


SCHEMATIC  VIEW 


Figure  14.  Perspective  and  schematic  views  of  a  low  fidelity,  laboratory 
simulation  for  studying  the  perceptual-motor  demands  of  laparoscopic  surgery. 
The  subject's  task  is  to  pick-up  plastic  chips  and  to  transfer  them  from  one 
container  to  another  using  Endo-Babcock  surgical  instruments.  Direct  viewing 
of  the  workspace  is  occluded.  Visual  feedback  is  provided  using  a  video  camera 
and  video  monitor. 
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Figure  15.  Schematic  view  showing  three  transfer  conditions.  Each  transfer 
condition  maintains  a  different  invariant  relation  to  the  training  condition  shown 
in  Figure  8.  Transfer  A  maintains  an  invariant  relation  between  the  hands  and 
the  work.  Transfer  B  maintains  an  invariant  relation  between  the  camera  and  the 
work.  Transfer  C  maintains  an  invariant  relation  between  the  camera  and  the 
hands. 
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Experiments  such  as  the  one  just  described  have  important  basic 
theoretical  and  practical  implications.  At  a  basic  theoretical  level,  the  question  is 
how  is  the  cognitive  representation  of  space  shaped  by  the  constraints  within  the 
workspace.  Which  coordinates  are  preserved  within  the  cognitive  representation 
—  the  intrinsic,  egocentric  body  coordinates  (right  and  left),  the  coordinates  in 
the  2-D  visual  representation  (i.e.,  video  screen),  the  extrinsic  coordinates  of 
absolute  3-D  space,  or  some  higher  order  combination  of  these  coordinate 
systems.  The  question  is  similar  to  the  questions  that  have  been  addressed  by 
Shepard  and  Metzler  (1971)  and  Kosslyn,  Ball  &  Reiser  (1987)  in  thier  classic 
work  on  visual  imagery.  The  unique  contribution  of  this  work  is  that  the 
questions  are  framed  within  an  interactive,  closed-loop,  perceptual  motor  task, 
whereas  much  of  the  previous  work  has  utilized  noninteractive,  perceptual 
judgment  tasks.  This  work  also  has  obvious  links  to  research  on  adaptation  to 
perceptual  rearrangement  (e.g.,  Welch,  1986)  and  mirror  tracing. 

In  addition  to  the  theoretical  issues,  this  research  has  obvious  implications 
for  the  design  and  utilization  of  simulators  for  surgical  training.  In  designing 
low  fidelity  simulators,  which  links  are  most  critical  for  ensuring  postitive 
transfer  to  surgery?  There  is  an  interesting  parallel  between  the  questions 
currently  being  raised  about  the  design  of  virtual  reality  systems  for  training 
advanced  surgical  procedures  and  questions  that  have  been  raised  in  the  past 
about  the  design  of  flight  simulators.  The  research  on  optical  flow  and  the 
control  of  locomotion  described  earlier  in  this  proposal  was  originally  motivated 
by  questions  about  the  design  of  low  fidelity  visual  flight  simulators. 


6.0  General  Summary 

This  report  describes  four  tracks  of  research  that  at  a  superficial 
level  appear  to  be  quite  disparate.  It  is  likely  that  some  will  view  this  as 
representative  of  a  piecemeal  and  disjointed  research  program.  Such  a  view 
highlights  the  differences  between  reductionistic  and  abstraction  strategies  of 
science.  Reductionistic  approaches  achieve  progress  by  studying  narrowly 
defined  problems  in  great  depth.  The  abstraction  strategy  achieves  progress  by 
discovering  invariants  across  different  systems.  The  goal  of  the  abstraction 
strategy  is  to  relate  the  invariants  to  fundamental  principles  that  will  have  broad 
generality.  We  believe  that  such  a  research  program  can  have  impact  at  both  a 
basic  science  and  an  applied  level. 

We  hope  that  it  will  be  apparent  to  the  careful  reader  that  the  problems  of 
controling  locomotion  using  information  in  optic  flow,  the  question  of  active 
versus  passive  observers,  the  problem  of  depth  perception,  and  the  problem  of 
manipulating  surgical  instruments  based  on  feedback  from  fiber  optic  cameras 
are  at  an  abstract  level  very  similar.  In  each  task  domain,  we  are  studying  how 
the  structure  of  information  and  the  dynamics  of  action  jointly  constrain 
coordinated  activity  and  situation  awareness. 
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We  believe  that  current  interest  in  situation  awareness  reflects  a  growing 
awareness  that  the  old  reductionistic  laboratory  strategies  based  on  stimulus- 
response  models  of  causation  are  inadequate.  Behavior  in  complex  work 
environments  reflects  a  coordination  across  a  wide  range  of  competing 
constraints.  Basic  research  programs  are  needed  that  acknowledge  and  address 
this  problem  of  coordination.  In  our  program  we  are  seeking  to  balance  the 
demands  for  experimental  rigor  with  the  need  for  a  holistic  perspective  that  is 
responsive  to  challenges  such  as  situation  awareness  that  reflect  the  real 
constraints  of  compex,  dynamic  work  environments. 

In  summary,  beyond  the  specific  questions  of  altitude  judgment,  dynamic 
occlusion,  depth  perception,  and  minimally  invasive  surgery,  we  believe  that  our 
program  illustrates  the  power  of  attacking  human  performance  as  a  control 
problem  of  coordinating  perception  and  action.  In  our  research  program  the 
independent  variables  have  been  constraints  on  information  (e.g.,  available 
texture,  observation  mode,  two-dimensional- video  image)  and  constraints  on 
action  (e.g.,  ascent  dynamics,  response  mode,  laparoscopic  instruments).  Our 
tasks  are  designed  so  that  we  can  measure  the  effects  of  these  kinds  of  constraints 
on  the  coordination  of  perception  and  action.  Where  possible,  we  have 
attempted  to  keep  the  closed-loop  coupling  between  perception  and  action  intact. 
In  generalizing  to  problems  such  as  situation  awareness,  we  believe  that  much 
progress  can  be  made  by  modeling  the  information  and  action  constraints  of  the 
task  environment.  These  constraints  constitute  the  behavioral  field.  As  such, 
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inhabit  the  field. 
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